Infrastructure

Components

The infrastructure for Lintol has a number of different components. Each of these components are hosted within a linux server environment. There are a numerous ways of hosting the linux environments required for Lintol. This documentation records and explains the infrastructure used by the Lintol development team.

Deployment Tools

Deploying the Lintol infrastructure on the components referenced in the Components section requires a number of deployment and configuration tools.

Application

Components

Lintol has a series of components that have been created using various technologies and programming languages. Distributed components connect via a central component named ‘Capstone’. Here is a list of the components, the general function they provide and the technology or programming language used to create them.

Diagram

alt text

Front-end

The Lintol front-end UI is the web interface component for Lintol. It provides a web portal for using Lintol’s other components. It is Javascript based project that uses Vue templates to render a single page application connected to the Lintol Capstone backend.

Front-end Deployment

Doorstep requires npm (node packet manager) to install the project dependencies. Here are the steps to run the front-end UI as a standalone process for development, production and testing.

# install dependencies
npm install

# serve with hot reload at localhost:8080
npm run dev

#build for production with minification
npm run build

# build for production and view the bundle analyzer report
npm run build --report

# run unit tests
npm run unit

# run e2e tests
npm run e2e

# run all tests
npm test

Doorstep

Doorstep is a CLI component of Lintol that runs the validation processors against the data provided. It can be operated via the command line as a standalone process or be integrated with the Lintol UI and Capstone components to provide a fully featured web application.

Doorstep creates data validation jobs by combining a data source with a validation script in a virtual environment that can be run in a single or multi threaded process. Standalone Doorstep will receive parameters for the script and data source and run a job in single threaded process.

Doorstep requires Python 3.4 or higher, geos package and virtualenv which can be installed using pip3. Here are the steps to run Doorstep as a standalone process from the command line.

Doorstep Deployment

OSX prerequisites

brew install python3
brew install geos
pip3 install virtualenv

Linux prerequisites

sudo apt-get install python3 python3-dev python3-venv libgeos libgeos-dev python3-pip

Get and run Doorstep

```shell git clone https://github.com/lintol/doorstep cd doorstep pip3 install -U setuptools python3 -m venv env . env/bin/activate python3 setup.py develop ltldoorstep process –engine dask.threaded example/data/protected_wrecks.geojson example/processors/boundary_checker_impr.py