mirror of https://github.com/gsi-upm/senpy
Macro commit
* Fixed Options for extra_params in UI * Enhanced meta-programming for models * Plugins can be imported from a python file if they're named `senpy_<whatever>.py>` (no need for `.senpy` anymore!) * Add docstings and tests to most plugins * Read plugin description from the docstring * Refactor code to get rid of unnecessary `.senpy`s * Load models, plugins and utils into the main namespace (see __init__.py) * Enhanced plugin development/experience with utils (easy_test, easy_serve) * Fix bug in check_template that wouldn't check objects * Make model defaults a private variable * Add option to list loaded plugins in CLI * Update docspre-1.0
J. Fernando Sánchez
6 years ago
parent
abd401f863
commit
21a5a3f201
@ -0,0 +1,113 @@
|
||||
Advanced plugin definition
|
||||
--------------------------
|
||||
In addition to finding plugins defined in source code files, senpy can also load a special type of definition file (`.senpy` files).
|
||||
This used to be the only mechanism for loading in earlier versions of senpy.
|
||||
|
||||
The definition file contains basic information
|
||||
|
||||
Lastly, it is also possible to add new plugins programmatically.
|
||||
|
||||
.. contents:: :local:
|
||||
|
||||
What is a plugin?
|
||||
=================
|
||||
|
||||
A plugin is a program that, given a text, will add annotations to it.
|
||||
In practice, a plugin consists of at least two files:
|
||||
|
||||
- Definition file: a `.senpy` file that describes the plugin (e.g. what input parameters it accepts, what emotion model it uses).
|
||||
- Python module: the actual code that will add annotations to each input.
|
||||
|
||||
This separation allows us to deploy plugins that use the same code but employ different parameters.
|
||||
For instance, one could use the same classifier and processing in several plugins, but train with different datasets.
|
||||
This scenario is particularly useful for evaluation purposes.
|
||||
|
||||
The only limitation is that the name of each plugin needs to be unique.
|
||||
|
||||
Definition files
|
||||
================
|
||||
|
||||
The definition file complements and overrides the attributes provided by the plugin.
|
||||
It can be written in YAML or JSON.
|
||||
The most important attributes are:
|
||||
|
||||
* **name**: unique name that senpy will use internally to identify the plugin.
|
||||
* **module**: indicates the module that contains the plugin code, which will be automatically loaded by senpy.
|
||||
* **version**
|
||||
* extra_params: to add parameters to the senpy API when this plugin is requested. Those parameters may be required, and have aliased names. For instance:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
extra_params:
|
||||
hello_param:
|
||||
aliases: # required
|
||||
- hello_param
|
||||
- hello
|
||||
required: true
|
||||
default: Hi you
|
||||
values:
|
||||
- Hi you
|
||||
- Hello y'all
|
||||
- Howdy
|
||||
|
||||
A complete example:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
name: <Name of the plugin>
|
||||
module: <Python file>
|
||||
version: 0.1
|
||||
|
||||
And the json equivalent:
|
||||
|
||||
.. code:: json
|
||||
|
||||
{
|
||||
"name": "<Name of the plugin>",
|
||||
"module": "<Python file>",
|
||||
"version": "0.1"
|
||||
}
|
||||
|
||||
|
||||
Example plugin with a definition file
|
||||
=====================================
|
||||
|
||||
In this section, we will implement a basic sentiment analysis plugin.
|
||||
To determine the polarity of each entry, the plugin will compare the length of the string to a threshold.
|
||||
This threshold will be included in the definition file.
|
||||
|
||||
The definition file would look like this:
|
||||
|
||||
.. code:: yaml
|
||||
|
||||
name: helloworld
|
||||
module: helloworld
|
||||
version: 0.0
|
||||
threshold: 10
|
||||
description: Hello World
|
||||
|
||||
Now, in a file named ``helloworld.py``:
|
||||
|
||||
.. code:: python
|
||||
|
||||
#!/bin/env python
|
||||
#helloworld.py
|
||||
|
||||
from senpy import AnalysisPlugin
|
||||
from senpy import Sentiment
|
||||
|
||||
|
||||
class HelloWorld(AnalysisPlugin):
|
||||
|
||||
def analyse_entry(entry, params):
|
||||
'''Basically do nothing with each entry'''
|
||||
|
||||
sentiment = Sentiment()
|
||||
if len(entry.text) < self.threshold:
|
||||
sentiment['marl:hasPolarity'] = 'marl:Positive'
|
||||
else:
|
||||
sentiment['marl:hasPolarity'] = 'marl:Negative'
|
||||
entry.sentiments.append(sentiment)
|
||||
yield entry
|
||||
|
||||
The complete code of the example plugin is available `here <https://lab.cluster.gsi.dit.upm.es/senpy/plugin-prueba>`__.
|
||||
@ -0,0 +1,13 @@
|
||||
This is a collection of plugins that exemplify certain aspects of plugin development with senpy.
|
||||
In ascending order of complexity, there are:
|
||||
|
||||
* Basic: a very basic analysis that does sentiment analysis based on emojis.
|
||||
* Configurable: a version of `basic` with a configurable map of emojis for each sentiment.
|
||||
* Parameterized: like `basic_info`, but users set the map in each query (via `extra_parameters`).
|
||||
* mynoop: shows how to add a definition file with external requirements for a plugin. Doing this with a python-only module would require moving all imports of the requirements to their functions, which is considered bad practice.
|
||||
* Async: a barebones example of training a plugin and analyzing data in parallel.
|
||||
|
||||
All of the plugins in this folder include a set of test cases and they are periodically tested with the latest version of senpy.
|
||||
|
||||
Additioanlly, for an example of stand-alone plugin that can be tested and deployed with docker, take a look at: lab.cluster.gsi.dit.upm.es/senpy/plugin-example
|
||||
bbm
|
||||
@ -1,8 +0,0 @@
|
||||
---
|
||||
name: Async
|
||||
module: asyncplugin
|
||||
description: I am async
|
||||
author: "@balkian"
|
||||
version: '0.1'
|
||||
async: true
|
||||
extra_params: {}
|
||||
@ -0,0 +1,23 @@
|
||||
#!/usr/local/bin/python
|
||||
# coding: utf-8
|
||||
|
||||
emoticons = {
|
||||
'marl:Positive': [':)', ':]', '=)', ':D'],
|
||||
'marl:Negative': [':(', ':[', '=(']
|
||||
}
|
||||
|
||||
emojis = {
|
||||
'marl:Positive': ['😁', '😂', '😃', '😄', '😆', '😅', '😄' '😍'],
|
||||
'marl:Negative': ['😢', '😡', '😠', '😞', '😖', '😔', '😓', '😒']
|
||||
}
|
||||
|
||||
|
||||
def get_polarity(text, dictionaries=[emoticons, emojis]):
|
||||
polarity = 'marl:Neutral'
|
||||
for dictionary in dictionaries:
|
||||
for label, values in dictionary.items():
|
||||
for emoticon in values:
|
||||
if emoticon and emoticon in text:
|
||||
polarity = label
|
||||
break
|
||||
return polarity
|
||||
@ -0,0 +1,40 @@
|
||||
#!/usr/local/bin/python
|
||||
# coding: utf-8
|
||||
|
||||
from senpy import easy_test, models, plugins
|
||||
|
||||
import basic
|
||||
|
||||
|
||||
class Basic(plugins.SentimentPlugin):
|
||||
'''Provides sentiment annotation using a lexicon'''
|
||||
|
||||
author = '@balkian'
|
||||
version = '0.1'
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
|
||||
polarity = basic.get_polarity(entry.text)
|
||||
|
||||
s = models.Sentiment(marl__hasPolarity=polarity)
|
||||
s.prov(self)
|
||||
entry.sentiments.append(s)
|
||||
yield entry
|
||||
|
||||
test_cases = [{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'So sad :(',
|
||||
'polarity': 'marl:Negative'
|
||||
}, {
|
||||
'input': 'Yay! Emojis 😁',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'But no emoticons 😢',
|
||||
'polarity': 'marl:Negative'
|
||||
}]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
easy_test()
|
||||
@ -0,0 +1,102 @@
|
||||
#!/usr/local/bin/python
|
||||
# coding: utf-8
|
||||
|
||||
from senpy import easy_test, models, plugins
|
||||
|
||||
import basic
|
||||
|
||||
|
||||
class Dictionary(plugins.SentimentPlugin):
|
||||
'''Sentiment annotation using a configurable lexicon'''
|
||||
|
||||
author = '@balkian'
|
||||
version = '0.2'
|
||||
|
||||
dictionaries = [basic.emojis, basic.emoticons]
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
polarity = basic.get_polarity(entry.text, self.dictionaries)
|
||||
|
||||
s = models.Sentiment(marl__hasPolarity=polarity)
|
||||
s.prov(self)
|
||||
entry.sentiments.append(s)
|
||||
yield entry
|
||||
|
||||
test_cases = [{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'So sad :(',
|
||||
'polarity': 'marl:Negative'
|
||||
}, {
|
||||
'input': 'Yay! Emojis 😁',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'But no emoticons 😢',
|
||||
'polarity': 'marl:Negative'
|
||||
}]
|
||||
|
||||
|
||||
class EmojiOnly(Dictionary):
|
||||
'''Sentiment annotation with a basic lexicon of emojis'''
|
||||
description = 'A plugin'
|
||||
dictionaries = [basic.emojis]
|
||||
|
||||
test_cases = [{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Neutral'
|
||||
}, {
|
||||
'input': 'So sad :(',
|
||||
'polarity': 'marl:Neutral'
|
||||
}, {
|
||||
'input': 'Yay! Emojis 😁',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'But no emoticons 😢',
|
||||
'polarity': 'marl:Negative'
|
||||
}]
|
||||
|
||||
|
||||
class EmoticonsOnly(Dictionary):
|
||||
'''Sentiment annotation with a basic lexicon of emoticons'''
|
||||
dictionaries = [basic.emoticons]
|
||||
|
||||
test_cases = [{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'So sad :(',
|
||||
'polarity': 'marl:Negative'
|
||||
}, {
|
||||
'input': 'Yay! Emojis 😁',
|
||||
'polarity': 'marl:Neutral'
|
||||
}, {
|
||||
'input': 'But no emoticons 😢',
|
||||
'polarity': 'marl:Neutral'
|
||||
}]
|
||||
|
||||
|
||||
class Salutes(Dictionary):
|
||||
'''Sentiment annotation with a custom lexicon, for illustration purposes'''
|
||||
dictionaries = [{
|
||||
'marl:Positive': ['Hello', '!'],
|
||||
'marl:Negative': ['sad', ]
|
||||
}]
|
||||
|
||||
test_cases = [{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'So sad :(',
|
||||
'polarity': 'marl:Negative'
|
||||
}, {
|
||||
'input': 'Yay! Emojis 😁',
|
||||
'polarity': 'marl:Positive'
|
||||
}, {
|
||||
'input': 'But no emoticons 😢',
|
||||
'polarity': 'marl:Neutral'
|
||||
}]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
easy_test()
|
||||
@ -0,0 +1,25 @@
|
||||
from senpy import AnalysisPlugin, easy
|
||||
|
||||
|
||||
class Dummy(AnalysisPlugin):
|
||||
'''This is a dummy self-contained plugin'''
|
||||
author = '@balkian'
|
||||
version = '0.1'
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
entry['nif:isString'] = entry['nif:isString'][::-1]
|
||||
entry.reversed = entry.get('reversed', 0) + 1
|
||||
yield entry
|
||||
|
||||
test_cases = [{
|
||||
'entry': {
|
||||
'nif:isString': 'Hello',
|
||||
},
|
||||
'expected': {
|
||||
'nif:isString': 'olleH'
|
||||
}
|
||||
}]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
easy()
|
||||
@ -1,11 +0,0 @@
|
||||
from senpy.plugins import SentimentPlugin
|
||||
|
||||
|
||||
class DummyPlugin(SentimentPlugin):
|
||||
def analyse_entry(self, entry, params):
|
||||
entry['nif:isString'] = entry['nif:isString'][::-1]
|
||||
entry.reversed = entry.get('reversed', 0) + 1
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
pass
|
||||
@ -1,15 +0,0 @@
|
||||
{
|
||||
"name": "Dummy",
|
||||
"module": "dummy",
|
||||
"description": "I am dummy",
|
||||
"author": "@balkian",
|
||||
"version": "0.1",
|
||||
"extra_params": {
|
||||
"example": {
|
||||
"@id": "example_parameter",
|
||||
"aliases": ["example", "ex"],
|
||||
"required": false,
|
||||
"default": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,27 +0,0 @@
|
||||
from senpy.plugins import SentimentPlugin
|
||||
|
||||
|
||||
class DummyNoInfo(SentimentPlugin):
|
||||
|
||||
description = 'This is a dummy self-contained plugin'
|
||||
author = '@balkian'
|
||||
version = '0.1'
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
entry['nif:isString'] = entry['nif:isString'][::-1]
|
||||
entry.reversed = entry.get('reversed', 0) + 1
|
||||
yield entry
|
||||
|
||||
test_cases = [{
|
||||
"entry": {
|
||||
"nif:isString": "Hello world!"
|
||||
},
|
||||
"expected": [{
|
||||
"nif:isString": "!dlrow olleH"
|
||||
}]
|
||||
}]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
d = DummyNoInfo()
|
||||
d.test()
|
||||
@ -1,2 +0,0 @@
|
||||
name: DummyNoInfo
|
||||
module: dummy_noinfo
|
||||
@ -1,14 +0,0 @@
|
||||
{
|
||||
"name": "DummyRequired",
|
||||
"module": "dummy",
|
||||
"description": "I am dummy",
|
||||
"author": "@balkian",
|
||||
"version": "0.1",
|
||||
"extra_params": {
|
||||
"example": {
|
||||
"@id": "example_parameter",
|
||||
"aliases": ["example", "ex"],
|
||||
"required": true
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,40 @@
|
||||
from senpy import AnalysisPlugin, easy
|
||||
|
||||
|
||||
class DummyRequired(AnalysisPlugin):
|
||||
'''This is a dummy self-contained plugin'''
|
||||
author = '@balkian'
|
||||
version = '0.1'
|
||||
extra_params = {
|
||||
'example': {
|
||||
'description': 'An example parameter',
|
||||
'required': True,
|
||||
'options': ['a', 'b']
|
||||
}
|
||||
}
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
entry['nif:isString'] = entry['nif:isString'][::-1]
|
||||
entry.reversed = entry.get('reversed', 0) + 1
|
||||
yield entry
|
||||
|
||||
test_cases = [{
|
||||
'entry': {
|
||||
'nif:isString': 'Hello',
|
||||
},
|
||||
'expected': None
|
||||
}, {
|
||||
'entry': {
|
||||
'nif:isString': 'Hello',
|
||||
},
|
||||
'params': {
|
||||
'example': 'a'
|
||||
},
|
||||
'expected': {
|
||||
'nif:isString': 'olleH'
|
||||
}
|
||||
}]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
easy()
|
||||
@ -0,0 +1,24 @@
|
||||
import noop
|
||||
from senpy.plugins import SentimentPlugin
|
||||
|
||||
|
||||
class NoOp(SentimentPlugin):
|
||||
'''This plugin does nothing. Literally nothing.'''
|
||||
|
||||
version = 0
|
||||
|
||||
def analyse_entry(self, entry, *args, **kwargs):
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
print(dir(noop))
|
||||
super(NoOp, self).test()
|
||||
|
||||
test_cases = [{
|
||||
'entry': {
|
||||
'nif:isString': 'hello'
|
||||
},
|
||||
'expected': {
|
||||
'nif:isString': 'hello'
|
||||
}
|
||||
}]
|
||||
@ -0,0 +1,3 @@
|
||||
module: mynoop
|
||||
requirements:
|
||||
- noop
|
||||
@ -1,5 +0,0 @@
|
||||
from senpy.plugins import SentimentPlugin
|
||||
|
||||
|
||||
class NoOp(SentimentPlugin):
|
||||
import noop
|
||||
@ -0,0 +1,63 @@
|
||||
#!/usr/local/bin/python
|
||||
# coding: utf-8
|
||||
|
||||
from senpy import easy_test, models, plugins
|
||||
|
||||
import basic
|
||||
|
||||
|
||||
class ParameterizedDictionary(plugins.SentimentPlugin):
|
||||
|
||||
description = 'This is a basic self-contained plugin'
|
||||
author = '@balkian'
|
||||
version = '0.2'
|
||||
|
||||
extra_params = {
|
||||
'positive-words': {
|
||||
'description': 'Comma-separated list of words that are considered positive',
|
||||
'aliases': ['positive'],
|
||||
'required': True
|
||||
},
|
||||
'negative-words': {
|
||||
'description': 'Comma-separated list of words that are considered negative',
|
||||
'aliases': ['negative'],
|
||||
'required': False
|
||||
}
|
||||
}
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
positive_words = params['positive-words'].split(',')
|
||||
negative_words = params['negative-words'].split(',')
|
||||
dictionary = {
|
||||
'marl:Positive': positive_words,
|
||||
'marl:Negative': negative_words,
|
||||
}
|
||||
polarity = basic.get_polarity(entry.text, [dictionary])
|
||||
|
||||
s = models.Sentiment(marl__hasPolarity=polarity)
|
||||
s.prov(self)
|
||||
entry.sentiments.append(s)
|
||||
yield entry
|
||||
|
||||
test_cases = [
|
||||
{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Positive',
|
||||
'parameters': {
|
||||
'positive': "Hello,:)",
|
||||
'negative': "sad,:()"
|
||||
}
|
||||
},
|
||||
{
|
||||
'input': 'Hello :)',
|
||||
'polarity': 'marl:Negative',
|
||||
'parameters': {
|
||||
'positive': "",
|
||||
'negative': "Hello"
|
||||
}
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
easy_test()
|
||||
@ -0,0 +1,27 @@
|
||||
from senpy.plugins import AnalysisPlugin
|
||||
from time import sleep
|
||||
|
||||
|
||||
class Sleep(AnalysisPlugin):
|
||||
'''Dummy plugin to test async'''
|
||||
author = "@balkian"
|
||||
version = "0.2"
|
||||
timeout = 0.05
|
||||
extra_params = {
|
||||
"timeout": {
|
||||
"@id": "timeout_sleep",
|
||||
"aliases": ["timeout", "to"],
|
||||
"required": False,
|
||||
"default": 0
|
||||
}
|
||||
}
|
||||
|
||||
def activate(self, *args, **kwargs):
|
||||
sleep(self.timeout)
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
sleep(float(params.get("timeout", self.timeout)))
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
pass
|
||||
@ -1,14 +0,0 @@
|
||||
from senpy.plugins import AnalysisPlugin
|
||||
from time import sleep
|
||||
|
||||
|
||||
class SleepPlugin(AnalysisPlugin):
|
||||
def activate(self, *args, **kwargs):
|
||||
sleep(self.timeout)
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
sleep(float(params.get("timeout", self.timeout)))
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
pass
|
||||
@ -1,16 +0,0 @@
|
||||
{
|
||||
"name": "Sleep",
|
||||
"module": "sleep",
|
||||
"description": "I am dummy",
|
||||
"author": "@balkian",
|
||||
"version": "0.1",
|
||||
"timeout": 0.05,
|
||||
"extra_params": {
|
||||
"timeout": {
|
||||
"@id": "timeout_sleep",
|
||||
"aliases": ["timeout", "to"],
|
||||
"required": false,
|
||||
"default": 0
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -0,0 +1,124 @@
|
||||
'''
|
||||
Meta-programming for the models.
|
||||
'''
|
||||
import os
|
||||
import json
|
||||
import jsonschema
|
||||
import inspect
|
||||
import copy
|
||||
|
||||
from abc import ABCMeta
|
||||
|
||||
|
||||
class BaseMeta(ABCMeta):
|
||||
'''
|
||||
Metaclass for models. It extracts the default values for the fields in
|
||||
the model.
|
||||
|
||||
For instance, instances of the following class wouldn't need to mark
|
||||
their version or description on initialization:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
class MyPlugin(Plugin):
|
||||
version=0.3
|
||||
description='A dull plugin'
|
||||
|
||||
|
||||
Note that these operations could be included in the __init__ of the
|
||||
class, but it would be very inefficient.
|
||||
'''
|
||||
_subtypes = {}
|
||||
|
||||
def __new__(mcs, name, bases, attrs, **kwargs):
|
||||
defaults = {}
|
||||
register_afterwards = False
|
||||
|
||||
attrs = mcs.expand_with_schema(name, attrs)
|
||||
if 'schema' in attrs:
|
||||
register_afterwards = True
|
||||
defaults = mcs.get_defaults(attrs['schema'])
|
||||
for b in bases:
|
||||
if hasattr(b, '_defaults'):
|
||||
defaults.update(b._defaults)
|
||||
info, attrs = mcs.split_attrs(attrs)
|
||||
defaults.update(info)
|
||||
attrs['_defaults'] = defaults
|
||||
|
||||
cls = super(BaseMeta, mcs).__new__(mcs, name, tuple(bases), attrs)
|
||||
|
||||
if register_afterwards:
|
||||
mcs.register(cls, cls._defaults['@type'])
|
||||
return cls
|
||||
|
||||
@classmethod
|
||||
def register(mcs, rsubclass, rtype=None):
|
||||
mcs._subtypes[rtype or rsubclass.__name__] = rsubclass
|
||||
|
||||
@staticmethod
|
||||
def expand_with_schema(name, attrs):
|
||||
if 'schema' in attrs: # Schema specified by name
|
||||
schema_file = '{}.json'.format(attrs['schema'])
|
||||
elif 'schema_file' in attrs:
|
||||
schema_file = attrs['schema_file']
|
||||
del attrs['schema_file']
|
||||
else:
|
||||
return attrs
|
||||
|
||||
if '/' not in 'schema_file':
|
||||
thisdir = os.path.dirname(os.path.realpath(__file__))
|
||||
schema_file = os.path.join(thisdir,
|
||||
'schemas',
|
||||
schema_file)
|
||||
|
||||
schema_path = 'file://' + schema_file
|
||||
|
||||
with open(schema_file) as f:
|
||||
schema = json.load(f)
|
||||
|
||||
resolver = jsonschema.RefResolver(schema_path, schema)
|
||||
attrs['@type'] = "".join((name[0].lower(), name[1:]))
|
||||
attrs['_schema_file'] = schema_file
|
||||
attrs['schema'] = schema
|
||||
attrs['_validator'] = jsonschema.Draft4Validator(schema, resolver=resolver)
|
||||
return attrs
|
||||
|
||||
@staticmethod
|
||||
def is_attr(k, v):
|
||||
return (not(inspect.isroutine(v) or
|
||||
inspect.ismethod(v) or
|
||||
inspect.ismodule(v) or
|
||||
isinstance(v, property)) and
|
||||
k[0] != '_' and
|
||||
k != 'schema' and
|
||||
k != 'data')
|
||||
|
||||
@staticmethod
|
||||
def split_attrs(attrs):
|
||||
'''
|
||||
Extract the attributes of the class.
|
||||
|
||||
This allows adding default values in the class definition.
|
||||
e.g.:
|
||||
'''
|
||||
isattr = {}
|
||||
notattr = {}
|
||||
for key, value in attrs.items():
|
||||
if BaseMeta.is_attr(key, value):
|
||||
if key[0] != '_':
|
||||
key = key.replace("__", ":", 1)
|
||||
isattr[key] = copy.deepcopy(value)
|
||||
else:
|
||||
notattr[key] = value
|
||||
return isattr, notattr
|
||||
|
||||
@staticmethod
|
||||
def get_defaults(schema):
|
||||
temp = {}
|
||||
for obj in [
|
||||
schema,
|
||||
] + schema.get('allOf', []):
|
||||
for k, v in obj.get('properties', {}).items():
|
||||
if 'default' in v and k not in temp:
|
||||
temp[k] = copy.deepcopy(v['default'])
|
||||
return temp
|
||||
@ -1,9 +0,0 @@
|
||||
---
|
||||
name: emoRand
|
||||
module: emoRand
|
||||
description: A sample plugin that returns a random emotion annotation
|
||||
author: "@balkian"
|
||||
version: '0.1'
|
||||
url: "https://github.com/gsi-upm/senpy-plugins-community"
|
||||
requirements: {}
|
||||
onyx:usesEmotionModel: "emoml:big6"
|
||||
@ -0,0 +1,32 @@
|
||||
import random
|
||||
|
||||
from senpy.plugins import EmotionPlugin
|
||||
from senpy.models import EmotionSet, Emotion, Entry
|
||||
|
||||
|
||||
class EmoRand(EmotionPlugin):
|
||||
'''A sample plugin that returns a random emotion annotation'''
|
||||
author = '@balkian'
|
||||
version = '0.1'
|
||||
url = "https://github.com/gsi-upm/senpy-plugins-community"
|
||||
onyx__usesEmotionModel = "emoml:big6"
|
||||
|
||||
def analyse_entry(self, entry, params):
|
||||
category = "emoml:big6happiness"
|
||||
number = max(-1, min(1, random.gauss(0, 0.5)))
|
||||
if number > 0:
|
||||
category = "emoml:big6anger"
|
||||
emotionSet = EmotionSet()
|
||||
emotion = Emotion({"onyx:hasEmotionCategory": category})
|
||||
emotionSet.onyx__hasEmotion.append(emotion)
|
||||
emotionSet.prov__wasGeneratedBy = self.id
|
||||
entry.emotions.append(emotionSet)
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
params = dict()
|
||||
results = list()
|
||||
for i in range(100):
|
||||
res = next(self.analyse_entry(Entry(nif__isString="Hello"), params))
|
||||
res.validate()
|
||||
results.append(res.emotions[0]['onyx:hasEmotion'][0]['onyx:hasEmotionCategory'])
|
||||
@ -1,10 +0,0 @@
|
||||
---
|
||||
name: rand
|
||||
module: rand
|
||||
description: A sample plugin that returns a random sentiment annotation
|
||||
author: "@balkian"
|
||||
version: '0.1'
|
||||
url: "https://github.com/gsi-upm/senpy-plugins-community"
|
||||
requirements: {}
|
||||
marl:maxPolarityValue: '1'
|
||||
marl:minPolarityValue: "-1"
|
||||
@ -1,33 +1,35 @@
|
||||
import random
|
||||
from senpy import SentimentPlugin, Sentiment, Entry
|
||||
|
||||
from senpy.plugins import SentimentPlugin
|
||||
from senpy.models import Sentiment, Entry
|
||||
|
||||
class Rand(SentimentPlugin):
|
||||
'''A sample plugin that returns a random sentiment annotation'''
|
||||
author = "@balkian"
|
||||
version = '0.1'
|
||||
url = "https://github.com/gsi-upm/senpy-plugins-community"
|
||||
marl__maxPolarityValue = '1'
|
||||
marl__minPolarityValue = "-1"
|
||||
|
||||
class RandPlugin(SentimentPlugin):
|
||||
def analyse_entry(self, entry, params):
|
||||
lang = params.get("language", "auto")
|
||||
|
||||
polarity_value = max(-1, min(1, random.gauss(0.2, 0.2)))
|
||||
polarity = "marl:Neutral"
|
||||
if polarity_value > 0:
|
||||
polarity = "marl:Positive"
|
||||
elif polarity_value < 0:
|
||||
polarity = "marl:Negative"
|
||||
sentiment = Sentiment({
|
||||
"marl:hasPolarity": polarity,
|
||||
"marl:polarityValue": polarity_value
|
||||
})
|
||||
sentiment["prov:wasGeneratedBy"] = self.id
|
||||
sentiment = Sentiment(marl__hasPolarity=polarity,
|
||||
marl__polarityValue=polarity_value)
|
||||
sentiment.prov(self)
|
||||
entry.sentiments.append(sentiment)
|
||||
entry.language = lang
|
||||
yield entry
|
||||
|
||||
def test(self):
|
||||
'''Run several random analyses.'''
|
||||
params = dict()
|
||||
results = list()
|
||||
for i in range(100):
|
||||
res = next(self.analyse_entry(Entry(nif__isString="Hello"), params))
|
||||
for i in range(20):
|
||||
res = next(self.analyse_entry(Entry(nif__isString="Hello"),
|
||||
params))
|
||||
res.validate()
|
||||
results.append(res.sentiments[0]['marl:hasPolarity'])
|
||||
assert 'marl:Positive' in results
|
||||
Loading…
Reference in New Issue