Contributing to Highcharts for Python

Note

As a general rule of thumb, the Highcharts for Python Toolkit applies PEP 8 styling, with some important differences.

Design Philosophy

Highcharts for Python is meant to be a “beautiful” and “usable” library. That means that it should offer an idiomatic API that:

  • works out of the box as intended,

  • minimizes “bootstrapping” to produce meaningful output, and

  • does not force users to understand how it does what it does.

In other words:

Users should simply be able to drive the car without looking at the engine.

The good news is that Highcharts (JS) applies a very similar philosophy, and so that makes the job for Highcharts for Python that much simpler.

Style Guide

Basic Conventions

  • Do not terminate lines with semicolons.

  • Line length should have a maximum of approximately 90 characters. If in doubt, make a longer line or break the line between clear concepts.

  • Each class should be contained in its own file.

  • If a file runs longer than 2,000 lines…it should probably be refactored and split.

  • All imports should occur at the top of the file - except where they have to occur inside a function/method to avoid circular imports or over-zealous soft dependency handling.

  • Do not use single-line conditions:

    # GOOD
if x:
  do_something()

# BAD
if x: do_something()

  • When testing if an object has a value, be sure to use if x is None: or if x is not None. Do not confuse this with if x: and if not x:.

  • Use the if x: construction for testing truthiness, and if not x: for testing falsiness. This is different from testing:

    • if x is True:

    • if x is False:

    • if x is None:

  • As of right now, we are using type annotations for function/method returns, but are not using type annotation for arguments consistently. This is because that would have a negative impact (we believe) on readability.

Naming Conventions

  • variable_name and not variableName or VariableName. Should be a noun that describes what information is contained in the variable. If a bool, preface with is_ or has_ or similar question-word that can be answered with a yes-or-no.

  • function_name and not functionName or FunctionName. Should be an imperative that describes what the function does (e.g. get_next_page).

  • CONSTANT_NAME and not constant_name or ConstantName.

  • ClassName and not class_name or Class_Name.

Basic Design Conventions

  • Functions at the module level can only be aware of objects either at a higher scope or singletons (which effectively have a higher scope).

  • Generally, functions and methods can use one positional argument (other than self or cls) without a default value. Any other arguments must be keyword arguments with default value given.

    def do_some_function(argument):
  # rest of function...

def do_some_function(first_arg,
                     second_arg = None,
                     third_arg = True):
  # rest of function ...

  • Functions and methods that accept values should start by validating their input, throwing exceptions as appropriate.

  • When defining a class, define all attributes in __init__.

  • When defining a class, start by defining its attributes and methods as private using a single-underscore prefix. Then, only once they’re implemented, decide if they should be public.

  • Don’t be afraid of the private attribute/public property/public setter pattern:

    class SomeClass(object):
  def __init__(*args, **kwargs):
    self._private_attribute = None

  @property
  def private_attribute(self):
    # custom logic which  may override the default return

    return self._private_attribute

  @setter.private_attribute
  def private_attribute(self, value):
    # custom logic that creates modified_value

    self._private_attribute = modified_value

  • Separate a function or method’s final (or default) return from the rest of the code with a blank line (except for single-line functions/methods).

  • Because Highcharts JS repeats many of the same properties and groups of properties, be sure to practice DRY. Use inheritance to your advantage, and don’t be afraid of the diamond of death inheritance problem.

    See also

Documentation Conventions

We are very big believers in documentation (maybe you can tell). To document Highcharts for Python we rely on several tools:

Sphinx

Sphinx is used to organize the library’s documentation into this lovely readable format (which is also published to ReadTheDocs [1]). This documentation is written in reStructuredText [2] files which are stored in <project>/docs.

Tip

As a general rule of thumb, we try to apply the ReadTheDocs [1] own Documentation Style Guide [3] to our RST documentation.

Hint

To build the HTML documentation locally:

  1. In a terminal, navigate to <project>/docs.

  2. Execute make html.

    Caution

    The Highcharts for Python documentation relies on Graphviz to render class inheritance diagrams. While in most Linux environments this should just work assuming it is installed, on Windows you will likley to have to use a more robust command to generate the full docs locally:

    $ sphinx-build -b html -D graphviz_dot="c:\Program Files\Graphviz\bin\dot.exe" . _build/html

    (and if necessary, adjust the location of dot.exe in your command)

When built locally, the HTML output of the documentation will be available at ./docs/_build/index.html.

Docstrings

  • Docstrings are used to document the actual source code itself. When writing docstrings we adhere to the conventions outlined in PEP 257.

Design Patterns and Standards

Highcharts is a large, robust, and complicated suite of JavaScript libraries. If in doubt, take a look at the extensive documentation and in particular the API reference. Because Highcharts for Python wraps the Highcharts Core API, its design is heavily shaped by Highcharts JS’ own design - as one should expect.

However, one of the main goals of Highcharts for Python is to make the Highcharts JS library a little more Pythonic to make it easier for Python developers to leverage it. Here are the notable design patterns that have been adopted that you should be aware of:

Code Style: Python vs JavaScript Naming Conventions

There are only two hard things in Computer Science: cache invalidation and naming things. – Phil Karlton

Highcharts Core is a JavaScript library, and as such it adheres to the code conventions that are popular (practically standard) when working in JavaScript. Chief among these conventions is that variables and object properties (keys) are typically written in camelCase.

A lot of (digital) ink has been spilled writing about the pros and cons of camelCase vs snake_case. While I have a scientific evidence-based opinion on the matter, in practice it is simply a convention that developers adopt in a particular programming language. The issue, however, is that while JavaScript has adopted the camelCase convention, Python generally skews towards the snake_case convention.

For most Python developers, using snake_case is the “default” mindset. Most of your Python code will use snake_case. So having to switch into camelcase to interact with Highcharts Core forces us to context switch, increases cognitive load, and is an easy place for us to overlook things and make a mistake that can be quite annoying to track down and fix later.

Therefore, when designing the Highcharts for Python Toolkit, we made several carefully considered design choices when it comes to naming conventions:

  1. All Highcharts for Python classes follow the Pythonic PascalCase class-naming convention.

  2. All Highcharts for Python properties and methods follow the Pythonic snake_case property/method/variable/function-naming convention.

  3. All inputs to properties and methods support both snake_case and camelCase (aka mixedCase) convention by default.

    This means that you can take something directly from Highcharts JavaScript code and supply it to the Highcharts for Python Toolkit without having to convert case or conventions. But if you are constructing and configuring something directly in Python using explicit deserialization methods, you can use snake_case if you prefer (and most Python developers will prefer).

    For example, if you supply a JSON file to a from_json() method, that file can leverage Highcharts (JS) natural camelCase convention OR Highcharts for Python’s snake_case convention.

    Warning

    Note that this dual-convention support only applies to deserialization methods and does not apply to the Highcharts for Python __init__() class constructors. All __init__() methods expect snake_case properties to be supplied as keywords.

  4. All outputs from serialization methods (e.g. to_dict() or to_js_literal()) will produce outputs that are Highcharts (JS)-compatible, meaning that they apply the camelCase convention.

Tip

Best Practice

If you are using external files to provide templates or themes for your Highcharts data visualizations, produce those external files using Highcharts JS’ natural camelCase convention. That will make it easier to re-use them elsewhere within a JavaScript context if you need to in the future.

Standard Methods: HighchartsMeta

Every single object supported by the Highcharts JS API corresponds to a Python class in Highcharts for Python. You can find the complete list in our comprehensive Highcharts for Python API Reference.

These classes generally inherit from the HighchartsMeta metaclass, which provides each class with a number of standard methods. These methods are the “workhorses” of Highcharts for Python and you will be relying heavily on them when using the library. Thankfully, their signatures and behavior is generally consistent - even if what happens “under the hood” is class-specific at times.

The standard methods exposed by the classes are:

Deserialization Methods

classmethod from_js_literal(cls, as_string_or_file, allow_snake_case=True)

Convert a JavaScript object defined using JavaScript object literal notation into a Highcharts for Python Python object, typically descended from HighchartsMeta.

Parameters:

  • cls (type) – The class object itself.

  • as_string_or_file (str) – The JavaScript object you wish to convert. Expects either a str containing the JavaScript object, or a path to a file which consists of the object.

  • allow_snake_case (bool) – If True, allows keys in as_string_or_file to apply the snake_case convention. If False, will ignore keys that apply the snake_case convention and only process keys that use the camelCase convention. Defaults to True.

Returns:

A Highcharts for Python object corresponding to the JavaScript object supplied in as_string_or_file.

Return type:

Descendent of HighchartsMeta

classmethod from_json(cls, as_json_or_file, allow_snake_case=True)

Convert a Highcharts JS object represented as JSON (in either str or bytes form, or as a file name) into a Highcharts for Python object, typically descended from HighchartsMeta.

Parameters:

  • cls (type) – The class object itself.

  • as_json_or_file (str or bytes) – The JSON object you wish to convert, or a filename that contains the JSON object that you wish to convert.

  • allow_snake_case (bool) – If True, allows keys in as_json to apply the snake_case convention. If False, will ignore keys that apply the snake_case convention and only process keys that use the camelCase convention. Defaults to True.

Returns:

A Highcharts for Python Python object corresponding to the JSON object supplied in as_json.

Return type:

Descendent of HighchartsMeta

classmethod from_dict(cls, as_dict, allow_snake_case=True)

Convert a dict representation of a Highcharts JS object into a Python object representation, typically descended from HighchartsMeta.

Parameters:

  • cls (type) – The class object itself.

  • as_dict (dict) – The dict representation of the object.

  • allow_snake_case (bool) – If True, allows keys in as_dict to apply the snake_case convention. If False, will ignore keys that apply the snake_case convention and only process keys that use the camelCase convention. Defaults to True.

Serialization Methods

to_js_literal(self, filename=None, encoding='utf-8')

Convert the Highcharts for Python instance to Highcharts JS-compatible JavaScript code using JavaScript object literal notation.

Parameters:

  • filename (Path-like or None) – If supplied, persists the JavaScript code to the file indicated. Defaults to None.

  • encoding (str) – Indicates the character encoding to use when producing the JavaScript literal string. Defaults to 'utf-8'.

Returns:

Highcharts JS-compatible JavaScript code using JavaScript object literal notation.

Return type:

str

to_json(self, filename=None, encoding='utf-8')

Convert the Highcharts for Python instance to Highcharts JS-compatible JSON.

Warning

While similar, JSON is inherently different from JavaScript object literal notation. In particular, it cannot include JavaScript functions. This means if you try to convert a Highcharts for Python object to JSON, any properties that are CallbackFunction instances will not be included. If you want to convert those functions, please use .to_js_literal() instead.

Parameters:

  • filename (Path-like or None) – If supplied, persists the JSON is persisted to the file indicated. Defaults to None.

  • encoding (str) – Indicates the character encoding to use when producing the JSON. Defaults to 'utf-8'.

Returns:

Highcharts JS-compatible JSON representation of the object.

Return type:

str or bytes

Note

Highcharts for Python works with different JSON encoders. If your environment has orjson, for example, the result will be returned as a bytes instance. Otherwise, the library will fallback to various other JSON encoders until finally falling back to the Python standard library’s JSON encoder/decoder.

to_dict(self)

Convert the Highcharts for Python object into a Highcharts JS-compatible dict object.

Returns:

Highcharts JS-compatible dict object

Return type:

dict

Other Convenience Methods

copy(self, other, overwrite=True, **kwargs)

Copy the properties from self to other.

Parameters:

  • other (HighchartsMeta) – The target instance to which the properties of this instance should be copied.

  • overwrite (bool) – if True, properties in other that are already set will be overwritten by their counterparts in self. Defaults to True.

  • kwargs – Additional keyword arguments. Some special descendents of HighchartsMeta may have special implementations of this method which rely on additional keyword arguments.

Returns:

A mutated version of other with new property values

Raises:

HighchartsValueError – if other is not the same class as (or subclass of) self

Module Structure

The structure of the Highcharts Core for Python library closely matches the structure of the Highcharts Core options object (see the relevant reference documentation).

At the root of the library - importable from highcharts_core you will find the highcharts_core.highcharts module. This module is a catch-all importable module, which allows you to easily access the most-commonly-used Highcharts for Python classes and modules.

Note

Whlie you can access all of the Highcharts Core for Python classes from highcharts_core.highcharts, if you want to more precisely navigate to specific class definitions you can do fairly easily using the folder organization and naming conventions used in the library.

This is the recommended best practice to maximize performance.

In the root of the highcharts_core library you can find universally-shared class definitions, like .metaclasses which contains the HighchartsMeta definition and the JavaScriptDict definition or .decorators which define method/property decorators that are used throughout the library.

The .utility_classes folder contains class definitions for classes that are referenced or used throughout the other class definitions.

And you can find the Highcharts Core options object and all of its properties defined in the .options module, with specific (complicated or extensive) sub-folders providing property-specific classes (e.g. the .options.plot_options module defines all of the different configuration options for different series types, while the .options.series module defines all of the classes that represent series of data in a given chart).

Class Structures and Inheritance

Highcharts for Python objects re-use many of the same properties. This is one of the strengths of the Highcharts API, in that it is internally consistent and that behavior configured on one object should be readily transferrable to a second object provided it shares the same properties. However, Highcharts has a lot of properties. We estimate that the options.plotOptions objects and their sub-properties have close to 3,000 properties. But because they are heavily repeated, those 3,000 or so properties can be reduced to only 421 unique property names. That’s almost an 85% reduction.

DRY is an important principle in software development. Can you imagine propagating changes in seven places (on average) in your code? That would be a maintenance nightmare! And it is exactly the kind of maintenance nightmare that class inheritance was designed to fix.

For that reason, the Highcharts for Python Toolkit classes have a deeply nested inheritance structure. This is important to understand both for evaluating isinstance() checks in your code, or for understanding how to further subclass Highcharts for Python components.

See also

For more details, please review the API documentation, in particular the class inheritance diagrams included for each documented class.

Multiple Inheritance, DRY and the Diamond of Death

Everything in moderation, including moderation. – Oscar Wilde

When contributing code to the Highcharts for Python Toolkit, it is important to understand how we handle multiple inheritance and the diamond of death problem.

First, obviously, multiple inheritance is generally considered an anti-pattern. That’s because it makes debugging code much, much harder - particuarly in Python, which uses a bit of a “magic” secret sauce called the MRO (Method Resolution Order) to determine which parent class’ methods to execute and when.

However, Highcharts - and by consequence, Highcharts for Python - is very verbose. We estimate that the full set of objects across the full Python toolkit has about 15,000 properties in total. A great many of these properties are identical in terms of their syntax, and their meaning (in context). So this is a classic example of where we can apply the principle of DRY to good effect. By using class inheritance, we can reduce the number of properties from about 15,000 to about 1,900. Not bad!

However, this significant reduction does require us to use multiple inheritance in some cases, paritcularly in the .options.series classes (which inherit from both the corresponding type-specific options in .options.plot_options) and from the generic SeriesBase class).

To solve the diamond of death problem, we implemented a number of private helper methods to assist in navigating the MRO:

Method / Function

Purpose

.utility_functions.get_remaining_mro()

Retrieve the class objects that are still to be traversed for a given class’ MRO.

.utility_functions.mro__to_untrimmed_dict()

Retrieve a consolidated untrimmed dict representation from all ancestors of a given class.

HighchartsMeta._untrimmed_mro_ancestors()

Method which consolidates the results of _to_untrimmed_dict() from a given instance’s parent class into a single dict.

HighchartsMeta._to_untrimmed_dict()

Generates an untrimmed dict representation of the instance at its lowest level in the class hierarchy. Think of this as the “bottom of the ladder”, with other methods (notably _untrimmed_mro_ancestors()) being used to generate corresponding dict from other rungs on the ladder.

When working on classes in the library:

  1. First, check whether the class has multiple inheritance. The easiest way to do this is to check the class inheritance diagram in the Highcharts Core for Python API Reference.

  2. Second, if a class you’re working on has mulitple inheritance, be sure to use the special functions and methods above as appropriate.

    Tip

    Best practice!

    Look at how we’ve implemented the standard methods for other classes with multiple inheritance. That will give you a good pattern to follow.

Dependencies

Note

Highcharts Core for Python has several types of dependencies:

  • “hard” dependencies, without which you will not be able to use the library at all,

  • “soft” dependencies, which will not produce errors but which may limit the value you get from the library,

  • “developer” dependencies that contributors will need in their local environment, and

  • “documentation” dependencies that are necessary if you wish to generate this documentation

Warning

If these hard dependencies are not available in the environment where Highcharts Core for Python is running, then the library will simply not work. Besides Highcharts Core (JS) itself, all of the other hard dependencies are automatically installed when installing Highcharts Core for Python using:

$ pip install highcharts-core

  • Highcharts Core v.10.2 or higher

    Note

    Not technically a Python dependency, but obviously Highcharts Core for Python will not work properly if your rendering layer does not leverage Highcharts Core (JS).

  • esprima-python v.4.0 or higher

  • requests v.2.31 or higher

  • validator-collection v.1.5 or higher

Preparing Your Development Environment

In order to prepare your local development environment, you should:

  1. Fork the Git repository.

  2. Clone your forked repository.

  3. Set up a virtual environment (optional).

  4. Install development dependencies:

highcharts-core/ $ pip install -r requirements.dev.txt

And you should be good to go!

Ideas and Feature Requests

Check for open issues or create a new issue to start a discussion around a bug or feature idea.

Testing

If you’ve added a new feature, we recommend you:

  • create local unit tests to verify that your feature works as expected, and

  • run local unit tests before you submit the pull request to make sure nothing else got broken by accident.

See also

For more information about the Highcharts for Python testing approach please see: Testing Highcharts for Python

Submitting Pull Requests

After you have made changes that you think are ready to be included in the main library, submit a pull request on Github and one of our developers will review your changes. If they’re ready (meaning they’re well documented, pass unit tests, etc.) then they’ll be merged back into the main repository and slated for inclusion in the next release.

Building Documentation

In order to build documentation locally, you can do so from the command line using:

highcharts-core/ $ cd docs
highcharts-core/docs $ make html

Caution

The Highcharts for Python documentation relies on Graphviz to render class inheritance diagrams. While in most Linux environments this should just work assuming it is installed, on Windows you will likley to have to use a more robust command to generate the full docs locally:

$ sphinx-build -b html -D graphviz_dot="c:\Program Files\Graphviz\bin\dot.exe" . _build/html

(and if necessary, adjust the location of dot.exe in your command)

When the build process has finished, the HTML documentation will be locally available at:

highcharts-core/docs/_build/html/index.html

Note

Built documentation (the HTML) is not included in the project’s Git repository. If you need local documentation, you’ll need to build it.

Contributors

Thanks to everyone who helps make Highcharts for Python useful:

References