Development¶

git clone git@github.com:openclimatedata/openscm.git
pip install -e .

Tests can be run locally with

python setup.py test

Writing a model adapter¶

Writing adapter tests¶

To help ensure your adapter works as intended, we provide a number of standard tests. To run these, create a file test_myadapter.py in tests/adapters/ and subclass the AdapterTester (this ensures that the standard tests are run on your adapter). Tests are done using pytest on all methods starting with test_. Only pull requests with adapters with full test coverage will be merged (see, for instance, the coverage on the end of the PR page).

# tests/adapters/test_myadapter.py

from openscm.adapters.myadapter import MyAdapter

from base import _AdapterTester


class TestMyAdapter(_AdapterTester):
    tadapter = MyAdapter

    # if necessary, you can extend the tests e.g.
    def test_run(self, test_adapter, test_run_parameters):
        super().test_run(test_adapter, test_run_parameters)
        # TODO some specific test of your adapter here

    def test_my_special_feature(self, test_adapter):
        # TODO test some special feature of your adapter class

Creating an `Adapter` subclass¶

Create your adapter source file in openscm/adapters/, e.g. myadapter.py, and subclass the openscm.adapter.Adapter class:

# openscm/adapters/myadapter.py

from ..adapter import Adapter

YEAR = 365 * 24 * 60 * 60  # example time step length as used below

class MyAdapter(Adapter):

Implement the relevant methods (or just do pass if you do not need to do anything in the particular method). The only part of OpenSCM with which adapters should interact is ParameterSet.

The _initialize_model() method initializes the adapter and is called only once just before the first call to the functions below initializing the first run. It should set the default values of mandatory model-specific (not standard OpenSCM parameters!) parameters in the in the ParameterSet stored in the adapter’s _parameters attribute. The hierarchical names of these model-specific parameters should start with the model/adapter name (as you set it in the model registry, see below).
```
def _initialize_model(self) -> None:
    # TODO Initialize the model
    # TODO Set default parameter values:
    self._parameters.get_writable_scalar_view(
        ("MyModel", "Specific Parameter"), ("World",), "Unit"
    ).set(DEFAULT_VALUE)
```
The _initialize_run_parameters() method initializes a particular run. It is called before the adapter is used in any way and at most once before a call to _run() or _step().
```
def _initialize_run_parameters(self) -> None:
    """
    TODO Initialize run parameters by reading model parameters
    from `self._parameters` (see below).
    """
```
The adapter should later use the start and stop time of the run as stored in the self._start_time and self._stop_time attributes.
The _initialize_model_input() method initializes the input and model parameters of a particular run. It is also called before the adapter is used in any way and at most once before a call to _run() or _step().

This and the _initialize_run_parameters() method are separated for higher efficiency when doing ensemble runs for models that have additional overhead for changing drivers/scenario setup.
```
def _initialize_model_input(self) -> None:
    """
    TODO Initialize model input by reading input parameters from
    :class:`self._parameters
    <~openscm.adapter.Adapter._parameters>` (see below).
    """
```
The _reset() method resets the model to prepare for a new run. It is called once after each call of _run() and to reset the model after several calls to _step().
```
def _reset(self) -> None:
    # TODO Reset the model
```

The _run() method runs the model over the full time range (as given by the times set by the previous call to _initialize_run_parameters()). You should at least implement this function.

def _run(self) -> None:
    """
    TODO Run the model and write output parameters to
    :class:`self._output <~openscm.adapter.Adapter._output>`
    (see below).
    """

The _step() method does a single time step. You can get the current time from self._current_time, which you should increase by the time step length and return its value. If your model does not support stepping just do raise NotImplementedError here.

def _step(self) -> None:
    """
    TODO Do a single time step and write corresponding output
    parameters to :class:`self._output
    <~openscm.adapter.Adapter._output>` (see below).
    """
    self._current_time += YEAR

The _shutdown() method cleans up the adapter.

def _shutdown(self) -> None:
    # TODO Shut down model

Reading model and input parameters and writing output parameters¶

Model parameters and input data (referred to as general “parameters” in OpenSCM) are pulled from the ParameterSet provided by the OpenSCM Core. OpenSCM defines a set of standard parameters to be shared between different SCMs. As far as possible, adapters should be able to take all of them as input from _parameters and should write their values to _output.

For efficiency, the OpenSCM Core interface provides subclasses of ParameterView that provide a view into a parameter with a requested time frame and unit. Conversion (aggregation, unit conversion, and time frame adjustment) is done interally if possible. Subclasses implement functionality for scalar and time series values, each for read-only as well as writable views, which you can get from the relevant ParameterSet (see Setting input parameters).

Accordingly, you should establish the views you need in the _initialize_model() method and save them as protected attributes of your adapter class. Then, get their values in the _initialize_model_input() and _initialize_run_parameters() methods. In the _run() and _step() methods you should write the relevant output parameters.

Adding the adapter to the model registry¶

Once done with your implementation, add a lookup for your adapter in openscm/adapters/__init__.py (where marked in the file) according to:

elif name == "MyAdapter":
    from .myadapter import MyAdapter

    adapter = MyAdapter

(make sure to set adapter to your class not an instance of your adapter)

Additional module dependencies¶

If your adapter needs additional dependencies add them to the REQUIREMENTS_MODELS dictionary in setup.py (see comment there).

Contributing¶

Thanks for contributing to OpenSCM. We are always trying to improve this tool, add new users and can do so even faster with your help!

Following the guidelines will help us work together as efficiently as possible. When we all work with a common understanding, we can make sure that issues are addressed quickly, suggested changes can be easily assessed and pull requests can be finalised painlessly.

All contributions are welcome, some possible suggestions include:

bug reports (make a new issue and use the template please :D)
feature requests (make a new issue and use the template please :D)
pull requests (make a pull request and use the template please :D)
tutorials (or support questions which, once solved, result in a new tutorial :D)
improving the documentation

Please don’t use the repository to have discussions about the results. Such discussions are scientific and generally belong in the scientific literature, not in a development repository.

Ground Rules¶

As a contributor, it is vital that we all follow a few conventions:

Be welcoming to newcomers and encourage diverse new contributors from all backgrounds. See the Code of Conduct.
Create issues for changes and enhancements, this ensures that everyone in the community has a chance to comment
Ensure that you pass all the tests before making a pull request
Avoid pushing directly to master, all changes should come via pull requests

Setup¶

Editor Config¶

The repository contains a .editorconfig file. This ensures that all of our text editors behave the same way and avoids spurious changes simply due to differing whitespace or end of line rules.

Many editors have built in support for Editorconfig but some require a plugin. To work out if your editor requires a plugin, please check https://editorconfig.org/.

Getting started¶

Your First Contribution¶

The development methodology for OpenSCM makes heavy use of git, make, virtual environments and test driven development. If you aren’t familiar with any of these terms it might be helpful to spend some time getting up to speed on these technologies. Some helpful resources (the longest take about 5 hours to work through):

Introduction to git by Software Carpentry
“Making a pull request”
“My first pull request”
Intoduction to tests by Software Carpentry and “Getting started with mocking”
Introduction to make by Software Carpentry
Virtual environments with venv
Continuous integration (CI); we use Travis CI for our CI but there are a number of good providers.
Jupyter Notebooks; we recommend simply installing jupyter (conda install jupyter or pip install jupyter) in your virtual environment.
Documentation generation with Sphinx

Development workflow¶

For almost all changes, there should be a corresponding Pull Request (PR) on GitHub to discuss the changes and track the overall implementation of the feature. These PRs should use the PR template.

It is better to break a larger problem into smaller features if you can. Each feature is implemented as a branch and merged into master once all of the tests pass. This is development workflow is preferred to one long-lived branch which can be difficult to merge.

The workflow for implementing a change to opencm is:

Create a PR. Initially you will not be ready to merge so prefix the title of the PR with ‘WIP:’.
Start a branch for the feature (or bug fix). When you start a new branch, be sure to pull any changes to master first.
```
git checkout master
git pull
git checkout -b my-feature
```
Develop your feature. Ensure that you run make test locally regularly to ensure that the tests still pass
Push your local development branch. This builds, tests and packages OpenSCM under Linux. The committer will be emailed if this process fails.
Before the PR can be merged it should be approved by another team member and it must pass the test suite. If you have a particular reviewer in mind, assign the PR to that user.
Your PR may need to be rebased before it can be merged. Rebasing replays your commits onto the new master commit and allows you to rewrite history.
```
git fetch
git checkout my-feature
git rebase -i origin/master
```
Once approved, a maintainer can merge the PR.

Testing¶

The tests are automatically run after every push using GitHub’s CI pipelines. If the tests fail, the person who committed the code is alerted via email.

Running the tests¶

To run the tests locally, simply run make test. This will create an isolated virtual environment with the required python libraries. This virtual environment can be manually regenerated using make venv -B.

Types of test¶

We have a number of different types of test:

unit, in the tests/unit folder
integration, in the tests/integration folder

Unit¶

Unit tests test isolated bits of code, one at a time. Thus, they only work if the tested functions are small and will almost inevitably require the use of mocking. Their purpose is to help to isolate bugs down to particular functions or lines of code.

Integration¶

Integration tests test a whole pipeline of functions on a higher level than unit tests. They ensure that all our joins make sense when run without (or with few) mocks. Overall, integration tests should reproduce how a user would interact with the package.

Release Process¶

We use tags to represent released versions of OpenSCM. Once you have tagged a new release in our git respoitory, versioneer takes care of the rest.

We follow Semantic Versioning, where version strings are of the format vMAJOR.MINOR.PATCH. We follow these conventions when deciding how to increment the version number, increment

MAJOR version when you make incompatible API changes,
MINOR version when you add functionality in a backwards-compatible manner
PATCH version when you make backwards-compatible bug fixes.

The steps undertaken to create a release are:

Checkout the latest commit in the master branch and ensure that your working copy is clean
Update CHANGELOG.rst to tag the unreleased items with the version and date of release. The unreleased section should now be empty.
Commit the changes with the message “Bumped to {}” where {} is replaced with the version string
Tag the commit with the version string. i.e. git tag v7.1.0
Push the commit and tags git push; git push --tags

Attribution¶

Thanks to https://github.com/nayafia/contributing-template/blob/master/CONTRIBUTING-template.md for the template.

Creating a release¶

OpenSCM uses designated Github Actions to upload the package to PyPI (and, in the future, also to Conda). To create a release:

Change the “master” header in CHANGELOG.rst to the release version number (not starting with “v”, e.g. “1.2.3”) and create a new, empty “master” header above. Commit these changes with the message, “Prepare for release of vVERSIONNUMBER’’ e.g. “Prepare for release of v1.2.3”.
Tag the commit as “vVERSIONNUMBER”, e.g. “v1.2.3”, on the “master” branch. Push the tag.
The Github Actions workflow should now create a release with the corresponding description in CHANGELOG.rst and upload the release to PyPI.

Code of Conduct¶

We as contributors and maintainers want to foster an open and welcoming environment around the OpenSCM project. To that end, we have a few ground rules that we ask everyone to adhere to. This code applies equally to everyone involved in the project.

This is not an exhaustive code which covers all possible behaviour. Rather, take it in the spirit in which it is intended - a guide to make it easier to enrich all of us and the technical communities in which we participate.

This code of conduct applies to all spaces managed by the OpenSCM project. This includes mailing lists, the issue tracker, group calls, and any other forums of the project. In addition, violations of this code outside these spaces may affect a person’s ability to participate within them.

If you believe someone is violating the code of conduct, we ask that you report it by emailing the maintainers listed in the README.

Be friendly and patient.
Be welcoming. We strive to be a community that welcomes and supports people of all backgrounds and identities. This includes, but is not limited to members of any race, ethnicity, culture, national origin, color, immigration status, social and economic class, educational level, sex, sexual orientation, gender identity and expression, age, size, family status, political belief, religion, and mental and physical ability.
Be considerate. Your work will be used by other people, and you in turn will depend on the work of others. Any decision you take will affect users and colleagues, and you should take those consequences into account when making decisions.
Be respectful. Not all of us will agree all the time, but disagreement is no excuse for poor behavior and poor manners. We might all experience some frustration now and then, but we cannot allow that frustration to turn into a personal attack. It is important to remember that a community where people feel uncomfortable or threatened is not a productive one. Everyone involved in the project should be respectful when dealing with others.
Be careful in the words that you choose. We want to be a community of professionals, and we conduct ourselves professionally. Be kind to others. Do not insult or put down other participants. Harassment and other exclusionary behavior are not acceptable. This includes, but is not limited to:
- Violent threats or language directed against another person.
- Discriminatory jokes and language.
- Posting sexually explicit or violent material.
- Posting (or threatening to post) other people’s personally identifying information (“doxing”).
- Personal insults, especially those using racist or sexist terms.
- Unwelcome sexual attention.
- Advocating for, or encouraging, any of the above behavior.
- Repeated harassment of others. In general, if someone asks you to stop, then stop.
When others disagree, try to understand why. Disagreements, both social and technical, happen all the time and this project is no exception. It is important that we resolve disagreements and differing views constructively. Different people have different perspectives on issues. Being unable to understand why someone holds a viewpoint does not mean that they are wrong. Do not forget that it is human to err and blaming each other does not get us anywhere. Instead, focus on helping to resolve issues and learning from mistakes.

This Code of Conduct is adapted from the Django Code of Conduct and the Contributor Covenant. Like these, this document is released under Creative Commons Attribution (CC BY)