salishsea.eos.ubc.ca Site Web App

The salishsea.eos.ubc.ca site is a dynamic web application written in Python 3 using the Pyramid web framework. These docs provide a brief explanation of the structure of the app, and details about its development.

If you are new to Python web frameworks you should probably start by reading the Pyramid Hello World docs, and then maybe proceed to the Quick Tutorial.

If you have experience with any of the many other Python web frameworks such as Django, flask, bottle, etc. you could start with the Quick Tutorial, or dive into the narrative documentation.

App Structure

The salishsea_site directory tree contains the app package:

The app configuration is defined in salishsea_site/__init__.py. That file also contains functions that map URL paths to route names. It also contains static view functions that serve the site static assets (see below) and model run results figures and other static files produced by the Salish Sea NEMO and SOG 1-D models.
The salishsea_site/views/ directory contains Python modules that define the view functions that the routes connect to. The view functions handle the HTTP requests coming into the app, and calculate the dynamic elements of the responses which are returned by the functions as dict objects. The view modules are organized to correspond to the site’s top-of-page navigation bar.
The salishsea_site/templates/ directory tree contains the page templates that the view functions pass to the renderer to produce the HTML stream that is sent to the user’s browser in the HTTP response. The dict objects returned by the view functions contain the template variable names and the values to substitute for them. The templates are written in HTML5. They use the Bootstrap 3 HTML/CSS/JS framework for layout and client-side dynamic elements. The Mako template library is used to provide variable substitution, inheritance, and flow control in the templates. The templates are organized in sub-directories that correspond to the view modules.
The salishsea_site/static/ directory tree contains the app’s static assets, CSS files, images, and PDF documents. They are served by a static view defined in salishsea_site/__init__.py.

The runtime configurations for running the app in a development environment, and for its production deployment are contained in the development.ini and production.ini files in the top level of the repository.

Please see Web App Package Development for details of how to set up an app development environment, and Running the App in Dev Mode for instructions on how to run the app in development mode on your local machine.

The tests/ directory tree contains the unit test suite for the package. It is intended to be run using the pytest tool. Please see Running the Unit Tests for details.

The docs/ directory tree contains the reStructuredText source files for these docs, and the Sphinx configuration and Makefile to render them to HTML. Please see Building the Documentation for details.

Web App Package Development

Python Versions

The salishsea-site package is developed, tested, and deployed using Python 3.10. The package uses some Python language features that are not available in versions prior to 3.6, in particular:

Formatted string literals (aka f-strings)
the file system path protocol

Getting the Code

Clone the salishsea-site code and documentation repository from GitHub with:

$ git clone git@github.com:SalishSeaCast/salishsea-site.git

or

$ git clone https://github.com/SalishSeaCast/salishsea-site.git

if you don’t have ssh key authentication set up on GitHub (or copy the link from the Clone or download button on the repository page).

Development Environment

Setting up an isolated development environment using Conda is strongly recommended. Assuming that you have Anaconda Python Distribution or Miniconda3 installed, you can create and activate an environment called salishsea-site that will have all of the Python packages necessary for development, testing, and building the documentation with the commands:

$ cd salishsea-site
$ conda env create -f env/environment-dev.yaml

The salishsea-site package is installed in editable install mode as part of the conda environment creation process. That means that the package is installed from the cloned repo via symlinks so that it will be automatically updated as the repo evolves.

To deactivate the environment use:

(salishsea-site)$ conda deactivate

Coding Style

The salishsea-site package uses the black code formatting tool to maintain a coding style that is very close to PEP 8.

black is installed as part of the Development Environment setup.

o run black on the entire code-base use:

$ cd SalishSeaCmd
$ conda activate salishsea-cmd
(salishsea-cmd)$ black ./

in the repository root directory. The output looks something like:

reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/mako_filters.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/site.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/about.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/bloomcast.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/tests/conftest.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/tests/test_mako_filters.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/tests/views/test_bloomcast.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/tests/views/test_figures.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/wwatch3.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/figures.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/__init__.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/fvcom.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/tests/views/test_salishseacast.py
reformatted /media/doug/warehouse/MEOPAR/salishsea-site/salishsea_site/views/salishseacast.py
All done! ✨ 🍰 ✨
14 files reformatted, 4 file left unchanged.

Running the App in Dev Mode

You can run the app in a development server on you local machine with the command:

(salishsea-site)$ cd salishsea-site
(salishsea-site)$ pserve --reload development.ini

With the dev server running you can view the site by navigating to http://localhost:6543/ in your browser.

The --reload option in the pserve command causes the dev server to monitor the app files in the salishsea-site/ directory tree and restart whenever it detects that a file has been changed. That allows you to easily edit app code and templates and instantly see changes in your browser after a page refresh.

Logger messages from the app appear in the terminal session where you ran the pserve command.

To stop the dev server use Ctrl-C in the terminal session where you ran the pserve command.

Note

Several pages in the app require access to parts of the /results/ directory tree on skookum. If you are working on a waterhole machine that has /results/ mounted, you are good to go. If you need to get /results/ mounted on a waterhole machine, please open an EOAS IT ticket with the request. If you are working on a remote machine or a laptop you can use sshfs to mount /results/ from skookum at a local /results/ mount point.

Building the Documentation

The documentation for the salishsea-site package is written in reStructuredText and converted to HTML using Sphinx.

If you have write access to the repository on GitHub, whenever you push changes to GitHub the documentation is automatically re-built and rendered at https://salishsea-site.readthedocs.io/en/latest/.

Additions, improvements, and corrections to these docs are always welcome.

The quickest way to fix typos, etc. on existing pages is to use the Edit on GitHub link in the upper right corner of the page to get to the online editor for the page on GitHub.

For more substantial work, and to add new pages, follow the instructions in the Development Environment section above. In the development environment you can build the docs locally instead of having to push commits to GitHub to trigger a build on readthedocs.org and wait for it to complete. Below are instructions that explain how to:

build the docs with your changes, and preview them in Firefox
check the docs for broken links

Building and Previewing the Documentation

Building the documentation is driven by the docs/Makefile. With your salishsea-site development environment activated, use:

(salishsea-site)$ (cd docs && make clean html)

to do a clean build of the documentation. The output looks something like:

Removing everything under '_build'...
Running Sphinx v3.0.1
making output directory... done
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from https://salishsea-meopar-docs.readthedocs.io/en/latest/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: [new config] 1 added, 0 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
generating indices...  genindexdone
writing additional pages...  searchdone
copying static files... ... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in _build/html.

The HTML rendering of the docs ends up in docs/_build/html/. You can open the index.html file in that directory tree in your browser to preview the results of the build before committing and pushing your changes to Bitbucket.

If you have write access to the repository on GitHub, whenever you push changes to GitHub the documentation is automatically re-built and rendered at https://salishsea-site.readthedocs.io/en/latest/.

Link Checking the Documentation

Sphinx also provides a link checker utility which can be run to find broken or redirected links in the docs. With your salishsea-site environment activated, use:

(salishsea-site)$ cd salishsea-site/docs/
(salishsea-site) docs$ make linkcheck

The output looks something like:

Running Sphinx v3.0.1
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [linkcheck]: targets for 1 source files that are out of date
updating environment: 0 added, 1 changed, 0 removed
reading sources... [100%] index
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] index
(line   21) ok        https://docs.python.org/3.8/
(line   21) ok        https://black.readthedocs.io/en/stable/
(line   21) ok        https://salishsea-site.readthedocs.io/en/latest/
(line   21) ok        https://github.com/SalishSeaCast/salishsea-site/actions?query=workflow%3Apytest-with-coverage
(line   21) ok        https://www.apache.org/licenses/LICENSE-2.0
(line   48) ok        https://salishsea.eos.ubc.ca/
(line   21) ok        https://github.com/SalishSeaCast/salishsea-site
(line   48) ok        https://www.python.org/
(line   21) ok        https://codecov.io/gh/SalishSeaCast/salishsea-site
(line   21) ok        https://github.com/SalishSeaCast/salishsea-site/issues
(line   48) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/index.html
(line   56) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/quick_tutorial/index.html
(line   56) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/firstapp.html#firstapp-chapter
(line   85) ok        https://docs.python.org/3/library/stdtypes.html#dict
(line   59) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/quick_tutorial/index.html
(line   59) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/index.html#narrative-documentation
(line   92) ok        https://developer.mozilla.org/en-US/docs/Web/Guide/HTML/HTML5
(line   92) ok        https://getbootstrap.com/
(line   85) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/views.html
(line   77) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/urldispatch.html
(line  152) ok        https://docs.python.org/3/reference/lexical_analysis.html#f-strings
(line  154) ok        https://docs.python.org/3/whatsnew/3.6.html#whatsnew36-pep519
(line   92) ok        https://docs.pylonsproject.org/projects/pyramid/en/latest/narr/templates.html
(line  183) ok        https://help.github.com/en/github/authenticating-to-github/connecting-to-github-with-ssh
(line   92) ok        https://www.makotemplates.org/
(line  118) ok        https://docs.pytest.org/en/latest/
(line  124) ok        https://www.sphinx-doc.org/en/master/usage/restructuredtext/basics.html
(line  169) ok        https://salishsea-meopar-docs.readthedocs.io/en/latest/repos_organization.html#salishsea-site-repo
(line  228) ok        https://www.python.org/dev/peps/pep-0008/
(line  194) ok        https://conda.io/en/latest/
(line  194) ok        https://salishsea-meopar-docs.readthedocs.io/en/latest/work_env/anaconda_python.html#anacondapythondistro
(line  316) ok        https://readthedocs.org/projects/salishsea-site/builds/
(line  194) ok        https://docs.conda.io/en/latest/miniconda.html
(line  518) ok        https://www.mercurial-scm.org/
(line   21) ok        https://img.shields.io/badge/license-Apache%202-cb2533.svg
(line  484) ok        https://coverage.readthedocs.io/en/latest/
(line  544) ok        https://salishsea.eos.ubc.ca/contributors
(line   21) ok        https://img.shields.io/badge/version%20control-git-blue.svg?logo=github
(line   21) ok        https://img.shields.io/badge/code%20style-black-000000.svg
(line   21) ok        https://img.shields.io/badge/python-3.8-blue.svg
(line   21) ok        https://readthedocs.org/projects/salishsea-site/badge/?version=latest
(line  512) ok        https://bitbucket.org/salishsea/salishsea-site/
(line  512) ok        https://img.shields.io/badge/version%20control-hg-blue.svg
(line   21) ok        https://codecov.io/gh/SalishSeaCast/salishsea-site/branch/main/graph/badge.svg
(line  518) ok        https://bitbucket.org/salishsea/salishsea-site
(line   21) ok        https://github.com/SalishSeaCast/salishsea-site/workflows/pytest-with-coverage/badge.svg
(line   21) ok        https://img.shields.io/github/issues/SalishSeaCast/salishsea-site?logo=github
(line  526) ok        https://img.shields.io/github/issues/SalishSeaCast/salishsea-site?logo=github
build succeeded.

Look for any errors in the above output or in _build/linkcheck/output.txt

make linkcheck is run monthly via a scheduled GitHub Actions workflow

Running the Unit Tests

The test suite for the salishsea-site package is in salishsea-site/tests/. The pytest tools is used for test fixtures and as the test runner for the suite.

With your salishsea-site development environment activated, use:

(salishsea-site)$ cd salishsea-site/
(salishsea-site)$ py.test

to run the test suite. The output looks something like:

============================ test session starts ============================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /media/doug/warehouse/MEOPAR/salishsea-site
collected 76 items
tests/test_mako_filters.py .......                                     [  9%]
tests/views/test_bloomcast.py ..                                       [ 11%]
tests/views/test_figures.py ...                                        [ 15%]
tests/views/test_salishseacast.py .....................................
...........................                                            [100%]
============================ 76 passed in 0.72s =============================

You can monitor what lines of code the test suite exercises using the coverage.py and pytest-cov tools with the command:

(salishsea-site)$ cd salishsea-site/
(salishsea-site)$ pytest --cov=./

The test coverage report will be displayed below the test suite run output.

Alternatively, you can use

(salishsea-site)$ pytest --cov=./ --cov-report html

to produce an HTML report that you can view in your browser by opening salishsea-site/htmlcov/index.html.

Continuous Integration

The salishsea-site package unit test suite is run and a coverage report is generated whenever changes are pushed to GitHub. The results are visible on the repo actions page, from the green checkmarks beside commits on the repo commits page, or from the green checkmark to the left of the “Latest commit” message on the repo code overview page . The testing coverage report is uploaded to codecov.io

The GitHub Actions workflow configuration that defines the continuous integration tasks is in the .github/workflows/pytest-coverage.yaml file.

Version Control Repository

The salishsea-site package code and documentation source files are available in the salishsea-site Git repository at https://github.com/SalishSeaCast/salishsea-site.

Issue Tracker

Development tasks, bug reports, and enhancement ideas are recorded and managed in the issue tracker at https://github.com/SalishSeaCast/salishsea-site/issues.

License

The salishsea.eos.ubc.ca site content, code, and documentation are Copyright 2014 – present by the Mesoscale Ocean and Atmospheric Dynamics (MOAD) group in the Department of Earth, Ocean, and Atmospheric Sciences at The University of British Columbia. Please see https://salishsea.eos.ubc.ca/contributors for details.

They are licensed under the Apache License, Version 2.0. https://www.apache.org/licenses/LICENSE-2.0 Please see the LICENSE file for details of the license.