Merge branch 'mesa'

docs/configuration.rst

@@ -1,241 +0,0 @@
-Configuring a simulation
-------------------------
-
-There are two ways to configure a simulation: programmatically and with a configuration file.
-In both cases, the parameters used are the same.
-The advantage of a configuration file is that it is a clean declarative description, and it makes it easier to reproduce.
-
-Simulation configuration files can be formatted in ``json`` or ``yaml`` and they define all the parameters of a simulation.
-Here's an example (``example.yml``).
-
-.. literalinclude:: example.yml
-   :language: yaml
-
-
-This example configuration will run three trials (``num_trials``) of a simulation containing a randomly generated network (``network_params``).
-The 100 nodes in the network will be SISaModel agents (``network_agents.agent_type``), which is an agent behavior that is included in Soil.
-10% of the agents (``weight=1``) will start in the content state, 10% in the discontent state, and the remaining 80% (``weight=8``) in the neutral state.
-All agents will have access to the environment (``environment_params``), which only contains one variable, ``prob_infected``.
-The state of the agents will be updated every 2 seconds (``interval``).
-
-Now run the simulation with the command line tool:
-
-.. code:: bash
-
-   soil example.yml
-
-Once the simulation finishes, its results will be stored in a folder named ``MyExampleSimulation``.
-Three types of objects are saved by default: a pickle of the simulation; a ``YAML`` representation of the simulation (which can be used to re-launch it); and for every trial, a ``sqlite`` file with the content of the state of every network node and the environment parameters at every step of the simulation.
-
-
-.. code::
-
-    soil_output
-    └── MyExampleSimulation
-        ├── MyExampleSimulation.dumped.yml
-        ├── MyExampleSimulation.simulation.pickle
-        ├── MyExampleSimulation_trial_0.db.sqlite
-        ├── MyExampleSimulation_trial_1.db.sqlite
-        └── MyExampleSimulation_trial_2.db.sqlite
-
-
-You may also ask soil to export the states in a ``csv`` file, and the network in gephi format (``gexf``).
-
-Network
-=======
-
-The network topology for the simulation can be loaded from an existing network file or generated with one of the random network generation methods from networkx.
-
-Loading a network
-#################
-
-To load an existing network, specify its path in the configuration:
-
-.. code:: yaml
-
-   ---
-   network_params:
-      path: /tmp/mynetwork.gexf
-
-Soil will try to guess what networkx method to use to read the file based on its extension.
-However, we only test using ``gexf`` files.
-
-For simple networks, you may also include them in the configuration itself using , using the ``topology`` parameter like so:
-
-.. code:: yaml
-
-   ---
-   topology:
-       nodes:
-          - id: First
-          - id: Second
-       links:
-          - source: First
-            target: Second
-
-
-Generating a random network
-###########################
-
-To generate a random network using one of networkx's built-in methods, specify the `graph generation algorithm <https://networkx.github.io/documentation/development/reference/generators.html>`_ and other parameters.
-For example, the following configuration is equivalent to :code:`nx.complete_graph(n=100)`:
-
-.. code:: yaml
-
-    network_params:
-        generator: complete_graph
-        n: 100
-
-Environment
-============
-The environment is the place where the shared state of the simulation is stored.
-For instance, the probability of disease outbreak.
-The configuration file may specify the initial value of the environment parameters:
-
-.. code:: yaml
-
-    environment_params:
-        daily_probability_of_earthquake: 0.001
-        number_of_earthquakes: 0
-
-All agents have access to the environment parameters.
-
-In some scenarios, it is useful to have a custom environment, to provide additional methods or to control the way agents update environment state.
-For example, if our agents play the lottery, the environment could provide a method to decide whether the agent wins, instead of leaving it to the agent.
-
-
-Agents
-======
-Agents are a way of modelling behavior.
-Agents can be characterized with two variables: agent type (``agent_type``) and state.
-Only one agent is executed at a time (generally, every ``interval`` seconds), and it has access to its state and the environment parameters.
-Through the environment, it can access the network topology and the state of other agents.
-
-There are three three types of agents according to how they are added to the simulation: network agents and environment agent.
-
-Network Agents
-##############
-Network agents are attached to a node in the topology.
-The configuration file allows you to specify how agents will be mapped to topology nodes.
-
-The simplest way is to specify a single type of agent.
-Hence, every node in the network will be associated to an agent of that type.
-
-.. code:: yaml
-
-   agent_type: SISaModel
-
-It is also possible to add more than one type of agent to the simulation, and to control the ratio of each type (using the ``weight`` property).
-For instance, with following configuration, it is five times more likely for a node to be assigned a CounterModel type than a SISaModel type.
-
-.. code:: yaml
-
-    network_agents:
-          - agent_type: SISaModel
-            weight: 1
-          - agent_type: CounterModel
-            weight: 5
-
-The third option is to specify the type of agent on the node itself, e.g.:
-
-
-.. code:: yaml
-
-   topology:
-       nodes:
-           - id: first
-   agent_type: BaseAgent
-   states:
-       first:
-         agent_type: SISaModel
-          
-
-This would also work with a randomly generated network:
-
-
-.. code:: yaml
-
-   network:
-       generator: complete
-       n: 5
-   agent_type: BaseAgent
-   states:
-       - agent_type: SISaModel
-
-              
-
-In addition to agent type, you may add a custom initial state to the distribution.
-This is very useful to add the same agent type with different states.
-e.g., to populate the network with SISaModel, roughly 10% of them with a discontent state:
-
-.. code:: yaml
-
-    network_agents:
-        - agent_type: SISaModel
-            weight: 9
-            state:
-            id: neutral
-        - agent_type: SISaModel
-            weight: 1
-            state:
-            id: discontent
-
-Lastly, the configuration may include initial state for one or more nodes.
-For instance, to add a state for the two nodes in this configuration:
-
-.. code:: yaml
-
-   agent_type: SISaModel
-   network:
-      generator: complete_graph
-      n: 2
-   states:
-     - id: content
-     - id: discontent
-
-
-Or to add state only to specific nodes (by ``id``).
-For example, to apply special skills to Linux Torvalds in a simulation:
-
-.. literalinclude:: ../examples/torvalds.yml
-   :language: yaml
-
-
-Environment Agents
-##################
-In addition to network agents, more agents can be added to the simulation.
-These agents are programmed in much the same way as network agents, the only difference is that they will not be assigned to network nodes.
-
-
-.. code::
-
-   environment_agents:
-       - agent_type: MyAgent
-         state:
-           mood: happy
-       - agent_type: DummyAgent
-
-
-You may use environment agents to model events that a normal agent cannot control, such as natural disasters or chance.
-They are also useful to add behavior that has little to do with the network and the interactions within that network.
-
-Templating
-==========
-
-Sometimes, it is useful to parameterize a simulation and run it over a range of values in order to compare each run and measure the effect of those parameters in the simulation.
-For instance, you may want to run a simulation with different agent distributions.
-
-This can be done in Soil using **templates**.
-A template is a configuration where some of the values are specified with a variable.
-e.g.,  ``weight: "{{ var1 }}"`` instead of ``weight: 1``.
-There are two types of variables, depending on how their values are decided:
-
-* Fixed. A list of values is provided, and a new simulation is run for each possible value. If more than a variable is given, a new simulation will be run per combination of values.
-* Bounded/Sampled. The bounds of the variable are provided, along with a sampler method, which will be used to compute all the configuration combinations.
-
-When fixed and bounded variables are mixed, Soil generates a new configuration per combination of fixed values and bounded values.
-
-Here is an example with a single fixed variable and two bounded variable:
-
-.. literalinclude:: ../examples/template.yml
-   :language: yaml

docs/example.yml

@@ -3,33 +3,38 @@ name: MyExampleSimulation
 max_time: 50
 num_trials: 3
 interval: 2
-network_params:
-    generator: barabasi_albert_graph
-    n: 100
-    m: 2
-network_agents:
-    - agent_type: SISaModel
-      weight: 1
+model_params:
+  topology:
+    params:
+      generator: barabasi_albert_graph
+      n: 100
+      m: 2
+  agents:
+    distribution:
+    - agent_class: SISaModel
+      topology: True
+      ratio: 0.1
       state:
-        id: content
-    - agent_type: SISaModel
-      weight: 1
+        state_id: content
+    - agent_class: SISaModel
+      topology: True
+      ratio: .1
       state:
-        id: discontent
-    - agent_type: SISaModel
-      weight: 8
+        state_id: discontent
+    - agent_class: SISaModel
+      topology: True
+      ratio: 0.8
       state:
-        id: neutral
-environment_params:
-    prob_infect: 0.075
-    neutral_discontent_spon_prob: 0.1
-    neutral_discontent_infected_prob: 0.3
-    neutral_content_spon_prob: 0.3
-    neutral_content_infected_prob: 0.4
-    discontent_neutral: 0.5
-    discontent_content: 0.5
-    variance_d_c: 0.2
-    content_discontent: 0.2
-    variance_c_d: 0.2
-    content_neutral: 0.2
-    standard_variance: 1
+        state_id: neutral
+  prob_infect: 0.075
+  neutral_discontent_spon_prob: 0.1
+  neutral_discontent_infected_prob: 0.3
+  neutral_content_spon_prob: 0.3
+  neutral_content_infected_prob: 0.4
+  discontent_neutral: 0.5
+  discontent_content: 0.5
+  variance_d_c: 0.2
+  content_discontent: 0.2
+  variance_c_d: 0.2
+  content_neutral: 0.2
+  standard_variance: 1

docs/index.rst

@@ -1,12 +1,20 @@
-.. Soil documentation master file, created by
-   sphinx-quickstart on Tue Apr 25 12:48:56 2017.
-   You can adapt this file completely to your liking, but it should at least
-   contain the root `toctree` directive.
-
 Welcome to Soil's documentation!
 ================================
 
-Soil is an Agent-based Social Simulator in Python focused on Social Networks.
+Soil is an opinionated Agent-based Social Simulator in Python focused on Social Networks.
+
+.. image:: soil.png
+  :width: 80%
+  :align: center
+
+Soil can be installed through pip (see more details in the :doc:`installation` page):
+
+.. code:: bash
+
+    pip install soil
+
+
+To get started developing your own simulations and agent behaviors, check out our :doc:`Tutorial <soil_tutorial>` and the `examples on GitHub <https://github.com/gsi-upm/soil/tree/master/examples>.
 
 If you use Soil in your research, do not forget to cite this paper:
 
@@ -38,8 +46,6 @@ If you use Soil in your research, do not forget to cite this paper:
    :caption: Learn more about soil:
 
    installation
-   quickstart
-   configuration
    Tutorial <soil_tutorial>
 
 ..

docs/installation.rst

@@ -1,7 +1,10 @@
 Installation
 ------------
 
-The easiest way to install Soil is through pip, with Python >= 3.4:
+Through pip
+===========
+
+The easiest way to install Soil is through pip, with Python >= 3.8:
 
 .. code:: bash
 
@@ -14,6 +17,10 @@ Now test that it worked by running the command line tool
 
    soil --help
 
+   #or
+
+   python -m soil --help
+
 Or, if you're using using soil programmatically:
 
 .. code:: python
@@ -21,4 +28,38 @@ Or, if you're using using soil programmatically:
    import soil
    print(soil.__version__)
 
-The latest version can be installed through `GitLab <https://lab.gsi.upm.es/soil/soil.git>`_ or `GitHub <https://github.com/gsi-upm/soil>`_.
+
+
+Web UI
+======
+
+Soil also includes a web server that allows you to upload your simulations, change parameters, and visualize the results, including a timeline of the network.
+To make it work, you have to install soil like this:
+
+.. code::
+
+  pip install soil[web]
+
+Once installed, the soil web UI can be run in two ways:
+
+.. code::
+
+  soil-web
+
+  # OR
+
+  python -m soil.web
+
+
+Development
+===========
+
+The latest version can be downloaded from `GitHub <https://github.com/gsi-upm/soil>`_ and installed manually:
+
+.. code:: bash
+
+   git clone https://github.com/gsi-upm/soil
+   cd soil
+   python -m venv .venv
+   source .venv/bin/activate
+   pip install --editable .

docs/make.bat

@@ -12,7 +12,7 @@ set BUILDDIR=_build
 set SPHINXPROJ=Soil
 
 if "%1" == "" goto help
-
+eE
 %SPHINXBUILD% >NUL 2>NUL
 if errorlevel 9009 (
 	echo.

docs/mesa.rst

@@ -0,0 +1,22 @@
+Mesa compatibility
+------------------
+
+Soil is in the process of becoming fully compatible with MESA.
+The idea is to provide a set of modular classes and functions that extend the functionality of mesa, whilst staying compatible.
+In the end, it should be possible to add regular mesa agents to a soil simulation, or use a soil agent within a mesa simulation/model.
+
+This is a non-exhaustive list of tasks to achieve compatibility:
+
+- [ ] Integrate `soil.Simulation` with mesa's runners:
+  - [ ] `soil.Simulation` could mimic/become a `mesa.batchrunner`
+- [ ] Integrate `soil.Environment` with `mesa.Model`:
+  - [x] `Soil.Environment` inherits from `mesa.Model`
+  - [x] `Soil.Environment` includes a Mesa-like Scheduler (see the `soil.time` module.
+  - [ ] Allow for `mesa.Model` to be used in a simulation.
+- [ ] Integrate `soil.Agent` with `mesa.Agent`:
+  - [x] Rename agent.id to unique_id?
+  - [x] mesa agents can be used in soil simulations (see `examples/mesa`)
+- [ ] Provide examples
+  - [ ] Using mesa modules in a soil simulation
+  - [ ] Using soil modules in a mesa simulation
+- [ ] Document the new APIs and usage

docs/notes_v1.0.rst

@@ -0,0 +1,35 @@
+What are the main changes in version 1.0?
+#########################################
+
+Version 1.0 is a major rewrite of the Soil system, focused on simplifying the API, aligning it with Mesa, and making it easier to use.
+Unfortunately, this comes at the cost of backwards compatibility.
+
+We drew several lessons from the previous version of Soil, and tried to address them in this version.
+Mainly:
+
+- The split between simulation configuration and simulation code was overly complicated for most use cases. As a result, most users ended up reusing configuration.
+- Storing **all** the simulation data in a database is costly and unnecessary for most use cases. For most use cases, only a handful of variables need to be stored. This fits nicely with Mesa's data collection system.
+- The API was too complex, and it was difficult to understand how to use it.
+- Most parts of the API were not aligned with Mesa, which made it difficult to use Mesa's features or to integrate Soil modules with Mesa code, especially for newcomers.
+- Many parts of the API were tightly coupled, which made it difficult to find bugs, test the system and add new features.
+
+The 0.30 rewrite should provide a middle ground between Soil's opinionated approach and Mesa's flexibility.
+The new Soil is less configuration-centric.
+It aims to provide more modular and convenient functions, most of which can be used in vanilla Mesa.
+
+How are agents assigned to nodes in the network
+###############################################
+
+The constructor of the `NetworkAgent` class has two arguments: `node_id` and `topology`.
+If `topology` is not provided, it will default to `self.model.topology`.
+This assignment might err if the model does not have a `topology` attribute, but most Soil environments derive from `NetworkEnvironment`, so they include a topology by default.
+If `node_id` is not provided, a random node will be selected from the topology, until a node with no agent is found.
+Then, the `node_id` of that node is assigned to the agent.
+If no node with no agent is found, a new node is automatically added to the topology.
+
+
+Can Soil environments include more than one network / topology?
+###############################################################
+
+Yes, but each network has to be included manually.
+Somewhere between 0.20 and 0.30 we included the ability to include multiple networks, but it was deemed too complex and was removed.

docs/quickstart.rst

@@ -1,93 +0,0 @@
-Quickstart
-----------
-
-This section shows how to run your first simulation with Soil.
-For installation instructions, see :doc:`installation`.
-
-There are mainly two parts in a simulation: agent classes and simulation configuration.
-An agent class defines how the agent will behave throughout the simulation.
-The configuration includes things such as number of agents to use and their type, network topology to use, etc.
-
-
-.. image:: soil.png
-  :width: 80%
-  :align: center
-
-
-Soil includes several agent classes in the ``soil.agents`` module, and we will use them in this quickstart.
-If you are interested in developing your own agents classes, see :doc:`soil_tutorial`.
-
-Configuration
-=============
-To get you started, we will use this configuration (:download:`download the file <quickstart.yml>` directly):
-
-.. literalinclude:: quickstart.yml
-   :language: yaml
-
-The agent type used, SISa, is a very simple model.
-It only has three states (neutral, content and discontent),
-Its parameters are the probabilities to change from one state to another, either spontaneously or because of contagion from neighboring agents.
-
-Running the simulation
-======================
-
-To see the simulation in action, simply point soil to the configuration, and tell it to store the graph and the history of agent states and environment parameters at every point.
-
-.. code::
-
-    ❯ soil --graph --csv quickstart.yml                                                          [13:35:29]
-    INFO:soil:Using config(s): quickstart
-    INFO:soil:Dumping results to soil_output/quickstart : ['csv', 'gexf']
-    INFO:soil:Starting simulation quickstart at 13:35:30.
-    INFO:soil:Starting Simulation quickstart trial 0 at 13:35:30.
-    INFO:soil:Finished Simulation quickstart trial 0 at 13:35:49 in 19.43677067756653 seconds
-    INFO:soil:Starting Dumping simulation quickstart trial 0 at 13:35:49.
-    INFO:soil:Finished Dumping simulation quickstart trial 0 at 13:35:51 in 1.7733407020568848 seconds
-    INFO:soil:Dumping results to soil_output/quickstart
-    INFO:soil:Finished simulation quickstart at 13:35:51 in 21.29862952232361 seconds
-
-
-The ``CSV`` file should look like this:
-
-.. code::
-
-   agent_id,t_step,key,value
-   env,0,neutral_discontent_spon_prob,0.05
-   env,0,neutral_discontent_infected_prob,0.1
-   env,0,neutral_content_spon_prob,0.2
-   env,0,neutral_content_infected_prob,0.4
-   env,0,discontent_neutral,0.2
-   env,0,discontent_content,0.05
-   env,0,content_discontent,0.05
-   env,0,variance_d_c,0.05
-   env,0,variance_c_d,0.1
-
-Results and visualization
-=========================
-
-The environment variables are marked as ``agent_id`` env.
-Th exported values are only stored when they change.
-To find out how to get every key and value at every point in the simulation, check out the :doc:`soil_tutorial`.
-
-The dynamic graph is exported as a .gexf file which could be visualized with
-`Gephi <https://gephi.org/users/download/>`__.
-Now it is your turn to experiment with the simulation.
-Change some of the parameters, such as the number of agents, the probability of becoming content, or the type of network, and see how the results change.
-
-
-Soil also includes a web server that allows you to upload your simulations, change parameters, and visualize the results, including a timeline of the network.
-To make it work, you have to install soil like this:
-
-.. code::
-
-  pip install soil[web]
-
-Once installed, the soil web UI can be run in two ways:
-
-.. code::
-
-  soil-web
-
-  # OR
-
-  python -m soil.web

docs/quickstart.yml

@@ -1,30 +0,0 @@
----
-name: quickstart
-num_trials: 1
-max_time: 1000
-network_agents:
-  - agent_type: SISaModel
-    state:
-      id: neutral
-    weight: 1
-  - agent_type: SISaModel
-    state:
-      id: content
-    weight: 2
-network_params:
-  n: 100
-  k: 5
-  p: 0.2
-  generator: newman_watts_strogatz_graph
-environment_params:
-    neutral_discontent_spon_prob: 0.05
-    neutral_discontent_infected_prob: 0.1
-    neutral_content_spon_prob: 0.2
-    neutral_content_infected_prob: 0.4
-    discontent_neutral: 0.2
-    discontent_content: 0.05
-    content_discontent: 0.05
-    variance_d_c: 0.05
-    variance_c_d: 0.1
-    content_neutral: 0.1
-    standard_variance: 0.1

docs/requirements.txt

@@ -1 +1 @@
-ipython==7.31.1
+ipython>=7.31.1

docs/soil-vs.rst

@@ -0,0 +1,12 @@
+### MESA
+
+Starting with version 0.3, Soil has been redesigned to complement Mesa, while remaining compatible with it.
+That means that every component in Soil (i.e., Models, Environments, etc.) can be mixed with existing mesa components.
+In fact, there are examples that show how that integration may be used, in the `examples/mesa` folder in the repository.
+
+Here are some reasons to use Soil instead of plain mesa:
+
+- Less boilerplate for common scenarios (by some definitions of common)
+- Functions to automatically populate a topology with an agent distribution (i.e., different ratios of agent class and state)
+- The `soil.Simulation` class allows you to run multiple instances of the same experiment (i.e., multiple trials with the same parameters but a different randomness seed)
+- Reporting functions that aggregate multiple