Doc changes

* Alabaster theme
* Restructured
* Simplified introduction
* Reference to entries/models
* Fixed examples

docs/api.rst

@@ -1,5 +1,5 @@
 NIF API
-=======
+-------
 .. http:get:: /api
 
    Basic endpoint for sentiment/emotion analysis.

docs/apischema.rst

@@ -0,0 +1,7 @@
+API and Schema
+##############
+.. toctree::
+
+   vocabularies.rst
+   api.rst
+   schema.rst

docs/architecture.rst

@@ -11,7 +11,5 @@ Senpy proposes a modular and dynamic architecture that allows:
 The framework consists of two main modules: Senpy core, which is the building block of the service, and Senpy plugins, which consist of the analysis algorithm. The next figure depicts a simplified version of the processes involved in an analysis with the Senpy framework.
 
 .. image:: senpy-architecture.png
-  :height: 400px
+  :width: 100%
-  :width: 800px
-  :scale: 100 %
   :align: center

docs/conf.py

@@ -37,6 +37,7 @@ extensions = [
     'sphinx.ext.todo',
     'sphinxcontrib.httpdomain',
     'sphinx.ext.coverage',
+    'sphinx.ext.autosectionlabel'
 ]
 
 # Add any paths that contain templates here, relative to this directory.
@@ -54,17 +55,21 @@ master_doc = 'index'
 # General information about the project.
 project = u'Senpy'
 copyright = u'2016, J. Fernando Sánchez'
+description = u'A framework for sentiment and emotion analysis services'
 
 # The version info for the project you're documenting, acts as replacement for
 # |version| and |release|, also used in various other places throughout the
 # built documents.
 #
 # The short X.Y version.
+# with open('../senpy/VERSION') as f:
+#     version = f.read().strip()
 # The full version, including alpha/beta/rc tags.
+# release = version
 
 # The language for content autogenerated by Sphinx. Refer to documentation
 # for a list of supported languages.
-#language = None
+language = None
 
 # There are two options for replacing |today|: either, you set today to some
 # non-false value, then it is used:
@@ -101,14 +106,14 @@ pygments_style = 'sphinx'
 #keep_warnings = False
 
 
+html_theme = 'alabaster'
 # -- Options for HTML output ----------------------------------------------
-if not on_rtd:  # only import and set the theme if we're building docs locally
+# if not on_rtd:  # only import and set the theme if we're building docs locally
-    import sphinx_rtd_theme
+#     import sphinx_rtd_theme
-    html_theme = 'sphinx_rtd_theme'
+#     html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
-    html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 
-else:
+# else:
-    html_theme = 'default'
+#     html_theme = 'default'
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
@@ -116,7 +121,13 @@ else:
 # Theme options are theme-specific and customize the look and feel of a theme
 # further.  For a list of options available for each theme, see the
 # documentation.
-#html_theme_options = {}
+html_theme_options = {
+    'logo': 'header.png',
+    'github_user': 'gsi-upm',
+    'github_repo': 'senpy',
+    'github_banner': True,
+}
+
 
 # Add any paths that contain custom themes here, relative to this directory.
 #html_theme_path = []
@@ -156,7 +167,13 @@ html_static_path = ['_static']
 #html_use_smartypants = True
 
 # Custom sidebar templates, maps document names to template names.
-#html_sidebars = {}
+html_sidebars = {
+    '**': [
+        'about.html',
+        'navigation.html',
+        'searchbox.html',
+        ]
+    }
 
 # Additional templates that should be rendered to pages, maps page names to
 # template names.

docs/demo.rst

@@ -1,7 +1,8 @@
 Demo
 ----
 
-There is a demo available on http://senpy.demos.gsi.dit.upm.es/, where you can a serie of different plugins. You can use them in the playground or make a directly requests to the service.
+There is a demo available on http://senpy.demos.gsi.dit.upm.es/, where you can test a serie of different plugins.
+You can use the playground (a web interface) or make HTTP requests to the service API.
 
 .. image:: senpy-playground.png
   :height: 400px

docs/index.rst

@@ -1,15 +1,28 @@
 Welcome to Senpy's documentation!
 =================================
 
-Contents:
+With Senpy, you can easily turn your sentiment or emotion analysis algorithm into a full blown semantic service.
+Sharing your sentiment analysis with the world has never been easier.
+Senpy provides:
+
+* Parameter validation, error handling
+* Formatting: JSON-LD, Turtle/n-triples input and output, or simple text input
+* Linked Data. Results are semantically annotated, using a series of well established vocabularies, and sane default URIs.
+* A web UI where users can explore your service and test different settings
+* A client to interact with any senpy service
+* A command line tool
+
+
 
 .. toctree::
+   :caption: Learn more about senpy
+   :maxdepth: 2
+
    senpy
    installation
    usage
-   api
+   apischema
-   schema
    plugins
    conversion
    demo
-   :maxdepth: 2
+   research.rst

docs/installation.rst

@@ -24,14 +24,33 @@ Docker Image
 ************
 Build the image or use the pre-built one:   
 
-    ``docker run -ti -p 5000:5000 gsiupm/senpy --host 0.0.0.0 --default-plugins``
+.. code:: bash
 
-To add custom plugins, add a volume and tell senpy where to find the plugins: 
+   docker run -ti -p 5000:5000 gsiupm/senpy --host 0.0.0.0 --default-plugins
+
+To add custom plugins, use a docker volume: 
     
-    ``docker run -ti -p 5000:5000 -v <PATH OF PLUGINS>:/plugins gsiupm/senpy --host 0.0.0.0 --default-plugins -f /plugins``
+.. code:: bash
 
+   docker run -ti -p 5000:5000 -v <PATH OF PLUGINS>:/plugins gsiupm/senpy --host 0.0.0.0 --default-plugins -f /plugins
 
-Note: There is a Senpy version for python2 too.
+Alias
+.....
+
+If you are using the docker approach regularly, it is advisable to use a script or an alias to simplify your executions:
+
+.. code:: bash
+
+   alias senpy='docker run --rm -ti -p 5000:5000 -v $PWD:/senpy-plugins gsiupm/senpy --default-plugins'
+ 
+
+Python 2
+........
+
+There is a Senpy version for python2 too:
     
-    ``docker run -ti -p 5000:5000 gsiupm/senpy-python2.7 --host 0.0.0.0 --default-plugins``
+.. code:: bash
+
+   docker run -ti -p 5000:5000 gsiupm/senpy:python2.7 --host 0.0.0.0 --default-plugins
+
 

docs/plugins.rst

@@ -2,29 +2,34 @@ Developing new plugins
 ----------------------
 This document describes how to develop a new analysis plugin. For an example of conversion plugins, see :doc:`conversion`.
 
-There is also a tutorial for developing new plugins with slides `here <https://lab.cluster.gsi.dit.upm.es/senpy/senpy-tutorial>`__ 
+A more step-by-step tutorial with slides is available `here <https://lab.cluster.gsi.dit.upm.es/senpy/senpy-tutorial>`__ 
 
-Each plugin represents a different analysis process.There are two types of files that are needed by senpy for loading a plugin:
+What is a plugin?
+=================
 
-- Definition file, has the ".senpy" extension.
+A plugin is a program that, given a text, will add annotations to it.
-- Code file, is a python file.
+In practice, a plugin consists of at least two files:
 
-This separation will allow us to deploy plugins that use the same code but employ different parameters.
+- Definition file: a `.senpy` file that describes the plugin (e.g. what input parameters it accepts, what emotion model it uses).
+- Python module: the actual code that will add annotations to each input.
+
+This separation allows us to deploy plugins that use the same code but employ different parameters.
 For instance, one could use the same classifier and processing in several plugins, but train with different datasets.
 This scenario is particularly useful for evaluation purposes.
 
 The only limitation is that the name of each plugin needs to be unique.
 
-Plugins Definitions
+Plugin Definition files
-===================
+=======================
 
 The definition file contains all the attributes of the plugin, and can be written in YAML or JSON.
+When the server is launched, it will recursively search for definition files in the plugin folder (the current folder, by default).
 The most important attributes are:
 
 * **name**: unique name that senpy will use internally to identify the plugin.
 * **module**: indicates the module that contains the plugin code, which will be automatically loaded by senpy.
 * **version**
-* extra_params: used to specify parameters that the plugin accepts that are not already part of the senpy API. Those parameters may be required, and have aliased names. For instance:
+* extra_params: to add parameters to the senpy API when this plugin is requested. Those parameters may be required, and have aliased names. For instance:
 
   .. code:: yaml
 
@@ -70,10 +75,28 @@ The basic methods in a plugin are:
 * __init__
 * activate: used to load memory-hungry resources
 * deactivate: used to free up resources
-* analyse_entry: called in every user requests. It takes in the parameters supplied by a user and should yield one or more ``Entry`` objects.
+* analyse_entry: called in every user requests. It takes two parameters: ``Entry``, the entry object, and ``params``, the parameters supplied by the user. It should yield one or more ``Entry`` objects.
 
 Plugins are loaded asynchronously, so don't worry if the activate method takes too long. The plugin will be marked as activated once it is finished executing the method.
 
+Entries
+=======
+
+Entries are objects that can be annotated.
+By default, entries are `NIF contexts <http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core/nif-core.html>`_ represented in JSON-LD format.
+Annotations are added to the object like this:
+
+.. code:: python
+
+   entry = Entry()
+   entry.vocabulary__annotationName = 'myvalue'
+   entry['vocabulary:annotationName'] = 'myvalue'
+   entry['annotationNameURI'] = 'myvalue'
+
+Where vocabulary is one of the prefixes defined in the default senpy context, and annotationURI is a full URI.
+The value may be any valid JSON-LD dictionary.
+For simplicity, senpy includes a series of models by default in the ``senpy.models`` module.
+
 
 Example plugin
 ==============
@@ -119,6 +142,13 @@ Now, in a file named ``helloworld.py``:
 
 F.A.Q.
 ======
+What annotations can I use?
+???????????????????????????
+
+You can add almost any annotation to an entry.
+The most common use cases are covered in the :doc:`schema`.
+
+
 Why does the analyse function yield instead of return?
 ??????????????????????????????????????????????????????
 

docs/research.rst

@@ -0,0 +1,11 @@
+Research
+--------
+
+If you use Senpy in your research, please cite `Senpy: A Pragmatic Linked Sentiment Analysis Framework <http://gsi.dit.upm.es/index.php/es/investigacion/publicaciones?view=publication&task=show&id=417>`__ (`BibTex <http://gsi.dit.upm.es/index.php/es/investigacion/publicaciones?controller=publications&task=export&format=bibtex&id=417>`__):
+
+.. code-block:: text
+
+	Sánchez-Rada, J. F., Iglesias, C. A., Corcuera, I., & Araque, Ó. (2016, October).
+	Senpy: A Pragmatic Linked Sentiment Analysis Framework.
+	In Data Science and Advanced Analytics (DSAA),
+	2016 IEEE International Conference on (pp. 735-742). IEEE.

docs/schema.rst

@@ -1,74 +1,74 @@
-Schema Examples
+Schema
-===============
+------
 All the examples in this page use the :download:`the main schema <_static/schemas/definitions.json>`.
 
 Simple NIF annotation
----------------------
+.....................
 Description
-...........
+,,,,,,,,,,,
 This example covers the basic example in the NIF documentation: `<http://persistence.uni-leipzig.org/nlp2rdf/ontologies/nif-core/nif-core.html>`_.
 
 Representation
-..............
+,,,,,,,,,,,,,,
-.. literalinclude:: examples/example-basic.json
+.. literalinclude:: examples/results/example-basic.json
    :language: json-ld
 
 Sentiment Analysis
----------------------
+.....................
 Description
-...........
+,,,,,,,,,,,
 
 Representation
-..............
+,,,,,,,,,,,,,,
 
-.. literalinclude:: examples/example-sentiment.json
+.. literalinclude:: examples/results/example-sentiment.json
    :emphasize-lines: 5-10,25-33
    :language: json-ld
 
 Suggestion Mining
------------------
+.................
 Description
-...........
+,,,,,,,,,,,
 
 Representation
-..............
+,,,,,,,,,,,,,,
 
-.. literalinclude:: examples/example-suggestion.json
+.. literalinclude:: examples/results/example-suggestion.json
    :emphasize-lines: 5-8,22-27
    :language: json-ld
 
 Emotion Analysis
-----------------
+................
 Description
-...........
+,,,,,,,,,,,
 
 Representation
-..............
+,,,,,,,,,,,,,,
 
-.. literalinclude:: examples/example-emotion.json
+.. literalinclude:: examples/results/example-emotion.json
    :language: json-ld
    :emphasize-lines: 5-8,25-37
 
 Named Entity Recognition
-------------------------
+........................
 Description
-...........
+,,,,,,,,,,,
 
 Representation
-..............
+,,,,,,,,,,,,,,
 
-.. literalinclude:: examples/example-ner.json
+.. literalinclude:: examples/results/example-ner.json
    :emphasize-lines: 5-8,19-34
    :language: json-ld
 
 Complete example
-----------------
+................
 Description
-...........
+,,,,,,,,,,,
 This example covers all of the above cases, integrating all the annotations in the same document.
 
 Representation
-..............
+,,,,,,,,,,,,,,
 
-.. literalinclude:: examples/example-complete.json
+.. literalinclude:: examples/results/example-complete.json
    :language: json-ld

docs/senpy.rst

@@ -1,18 +1,32 @@
 What is Senpy?
 --------------
 
-Senpy is a framework to build semantic sentiment and emotion analysis services. Using Senpy you can easy develop and publish your own analysis algorithms.
+Senpy is a framework that turns your sentiment or emotion analysis algorithm into a full blown semantic service.
+Senpy takes care of:
 
-If you use Senpy, please cite `Senpy: A Pragmatic Linked Sentiment Analysis Framework <http://gsi.dit.upm.es/index.php/es/investigacion/publicaciones?view=publication&task=show&id=417>`__ (`BibTex <http://gsi.dit.upm.es/index.php/es/investigacion/publicaciones?controller=publications&task=export&format=bibtex&id=417>`__):
+  * Interfacing with the user: parameter validation, error handling.
+  * Formatting: JSON-LD, Turtle/n-triples input and output, or simple text input
+  * Linked Data: senpy results are semantically annotated, using a series of well established vocabularies, and sane default URIs.
+  * User interface: a web UI where users can explore your service and test different settings
+  * A client to interact with the service. Currently only available in Python.
 
-.. code-block:: text
+Sharing your sentiment analysis with the world has never been easier!
+
+Senpy for service developers
+============================
+
+Check out the :doc:`plugins` if you have developed an analysis algorithm (e.g. sentiment analysis) and you want to publish it as a service.
+
+Senpy for end users
+===================
+
+All services built using senpy share a common interface.
+This allows users to use them (almost) interchangeably.
+Senpy comes with a :ref:`built-in client`.
 
-	Sánchez-Rada, J. F., Iglesias, C. A., Corcuera, I., & Araque, Ó. (2016, October).
-	Senpy: A Pragmatic Linked Sentiment Analysis Framework.
-	In Data Science and Advanced Analytics (DSAA),
-	2016 IEEE International Conference on (pp. 735-742). IEEE.
 
 .. toctree::
-   	specifications
+    :caption: Interested? Check out senpy's:
-   	architecture
+ 
+    architecture
 

docs/usage.rst

@@ -1,20 +1,9 @@
 Usage
 -----
 
-The easiest and recommended way is to just use the command-line tool to load your plugins and launch the server.
+First of all, you need to install the package.
-
+See :doc:`installation` for installation instructions.
-.. code:: bash
+Once installed, the `senpy` command should be available. 
-
-   senpy
-
-Or, alternatively:
-
-.. code:: bash
-
-    python -m senpy
-
-
-This will create a server with any modules found in the current path.
 
 Useful command-line options
 ===========================
@@ -23,19 +12,19 @@ In case you want to load modules, which are located in different folders under t
 
 .. code:: bash
 
-    python -m senpy -f .
+    senpy -f .
 
-The default port used by senpy is 5000, but you can change it using the option `--port`.
+The default port used by senpy is 5000, but you can change it using the `--port` flag.
 
 .. code:: bash
 
-    python -m senpy --port 8080
+    senpy --port 8080
 
 Also, the host can be changed where senpy is deployed. The default value is `127.0.0.1`.
 
 .. code:: bash
 
-    python -m senpy --host 0.0.0.0
+    senpy --host 0.0.0.0
 
 For more options, see the `--help` page.
 
@@ -48,15 +37,19 @@ Once the server is launched, there is a basic endpoint in the server, which prov
 
 In case you want to know the different endpoints of the server, there is more information available in the NIF API section_.
 
-CLI
+CLI demo
-===
+========
 
 This video shows how to use senpy through command-line tool.
 
-https://asciinema.org/a/9uwef1ghkjk062cw2t4mhzpyk
+.. image:: https://asciinema.org/a/9uwef1ghkjk062cw2t4mhzpyk.png
+   :width: 100%
+   :target: https://asciinema.org/a/9uwef1ghkjk062cw2t4mhzpyk
+   :alt: CLI demo
 
-Request example in python
+
-=========================
+Built-in client
+===============
 
 This example shows how to make a request to the default plugin:
 

docs/vocabularies.rst

@@ -1,7 +1,7 @@
-Specifications
+Vocabularies and model
-==============
+======================
 
-The model used in Senpy is based on the following specifications:
+The model used in Senpy is based on the following vocabularies:
 
 * Marl, a vocabulary designed to annotate and describe subjetive opinions expressed on the web or in information systems.
 * Onyx, which is built one the same principles as Marl to annotate and describe emotions, and provides interoperability with Emotion Markup Language.