Welcome to c2cgeoform’s documentation!¶

Prerequisites¶

The following system packages must be installed on your system:

python-virtualenv
libpq-dev (header files for PostgreSQL)
gettext

On Windows, you should install make using Cygwin (and put the bin folder into the path). For Python, please use Python >= 3.x.

You need to install PostgreSQL and PostGIS. On Ubuntu, the packages postgresql-server-dev-9.3 and python-dev are required.

User guide¶

Creating a c2cgeoform project¶

This page describes how to create a c2cgeoform project. A c2cgeoform project is basically a Pyramid project with c2cgeoform enabled in the project.

Install c2cgeoform¶

git clone git@github.com:camptocamp/c2cgeoform.git
cd c2cgeoform
make build

On Windows, you should use the https way to clone the repository:

git clone https://github.com:camptocamp/c2cgeoform.git

Create a Pyramid project using c2cgeoform scaffold¶

Note that if PYTHONPATH does not exists as an environment variable, template files (*_tmpl) are not rendered in new project folder.

export PYTHONPATH=$PYTHONPATH
.build/venv/bin/pcreate -s c2cgeoform ../c2cgeoform_project

Initialize a git repository¶

Make your new project folder a git repository.

cd ../c2cgeoform_project
git init
git add .
git commit -m 'Initial commit'

Install the project and its dependencies¶

make build

Set up database¶

First of all you need to have a PostGIS database for the project. Create the database:

sudo -u postgres psql -c "CREATE USER \"www-data\" WITH PASSWORD 'www-data';"

sudo -u postgres createdb c2cgeoform_project
sudo -u postgres psql -d c2cgeoform_project -c 'CREATE EXTENSION postgis;'
sudo -u postgres psql -c 'GRANT ALL ON DATABASE c2cgeoform_project TO "www-data";'

When you do have a Postgres role and a PostGIS database edit the development.ini and production.ini files and set sqlachemy.url appropriately. For example:

sqlalchemy.url = postgresql://www-data:www-data@localhost:5432/c2cgeoform_project

Now create the tables:

make initdb

Note that this will launch the python script c2cgeoform_project/scripts/initializedb.py. You will have to customize this thereafter.

Run the development server¶

You are now ready to run the application:

make serve

Visit the following URLs to verify that the application works correctly: http://localhost:6543/excavations/new and http://localhost:6543/excavations.

Defining the model for a form¶

The underlying schema for a c2cgeoform form is defined as a SQLAlchemy model. A simple definition is shown below:

from sqlalchemy import (Column, Integer, Text)
import deform
from uuid import uuid4

from c2cgeoform.models import Base


class Comment(Base):
    __tablename__ = 'comments'
    __colanderalchemy_config__ = {
        'title': 'A very simple form'
    }

    id = Column(Integer, primary_key=True, info={
        'colanderalchemy': {
            'widget': deform.widget.HiddenWidget()
        }})

    hash = Column(Text, unique=True, default=lambda: str(uuid4(), info={
        'colanderalchemy': {
            'widget': HiddenWidget()
        }})

    name = Column(Text, nullable=False, info={
        'colanderalchemy': {
            'title': 'Name'
        }})

    comment = Column(Text, nullable=True, info={
        'colanderalchemy': {
            'title': 'Comment',
            'widget': deform.widget.TextAreaWidget(rows=3),
        }})

This SQLAlchemy model is enriched with properties for ColanderAlchemy, for example to set a title for a field, use a specific Deform widget or use a Colander validator.

In general, every SQLAlchemy model can be used as schema for a form. The only requirements are:

The model class must contain exactly one primary key column. Tables with composite primary keys are not supported.

A more complex example for a model can be found here. For more information on how to define the model, please refer to the SQLAlchemy, ColanderAlchemy, Colander and Deform documentations.

Create the views for your model¶

There is already a views class created in your project by the scaffold, see file views/excavation.py. Let’s have a look on that file content.

To ease creation of views classes, c2cgeoform comes with an abstract class that contains base methods to display grids, render forms and save data. This is why ExcavationViews extends AbstractViews for a specific SQLAlchemy model and colander schema:

@view_defaults(match_param='table=excavations')
class ExcavationViews(AbstractViews):

    _model = Excavation
    _base_schema = base_schema

Also note the @view_defaults which says that all the views declared in this class will only apply when the route parameter named table will be equal to "excavation". The routes given by c2cgeoform have the following form:

c2cgeoform_index: {table}
c2cgeoform_grid: {table}/grid.json
c2cgeoform_item: {table}/{{id}}
c2cgeoform_item_duplicate: {table}/{{id}}/duplicate

Those routes are registered in the pyramid config by the routes module (see the routes.py file situated at the root of the generated project).

register_models(config, [
    ('excavations', Excavation)

To select records through urls, we also need a unique field, this is given by:

_id_field = 'hash'

And to show the table records grid we need a definition per column:

_list_fields = [
    _list_field('reference_number'),
    _list_field('request_date'),
    ...
]

Finally we need a method for each view, for a typical use case, we could have 6 views:

index: Return HTML page with the grid.
grid: Return records as JSON for the grid.
edit: Show create or edit form for the specified record.
duplicate: Show duplication form for the specified record.
delete: Delete the specified record.
save: Save new record or modifications to existing record.

In a typical use case, those views will only call the super class method with the same name.

Configure the grid¶

Grid columns can be configured using the _list_fields property of the views class, which is an ordered list of ListField objects, one for each column.

The ListField constructor take some parameters:

model: the SQLAlchemy mapper (required if attr is an attribute name).
attr: the model attribute name to use or an SQLAlchemy InstrumentedAttribute.
key: an identifier for the column, default to attribute.key.
label: text for the column header, default to colanderalchemy title for the field.
renderer: callable that takes an entity of the SQLAlchemy mapper and returns a string value.
sort_column: An IntrumentedAttribute to use in sort_by.
filter_column: An IntrumentedAttribute to filter with.
visible: a boolean for the initial visible state of this column.

Every time the table index page asks for data from the grid view, the AbstractView will create a default query using AbstractViews._base_query method.

If you use columns coming from relationships, this might result in sending one request to the database for each relationship and each record. In such cases, you should override the _base_query method to use eager loading for those relationships, for example:

def _base_query(self):
    return self._request.dbsession.query(Excavation).distinct(). \
        join('situations'). \
        options(subqueryload('situations'))

Note that you also need to join the relationships you use for sorting and filtering.

Understanding the schemas¶

ColanderAlchemy allows creating Colander schemas directly from SQLAlchemy model classes.

Additionally, c2cgeoform provides its own classes with extended features. A basic use case schema creation will look like:

from model import MyClass
schema = GeoFormSchemaNode(MyClass)

See the following API to understand what is going on behind the scene.

class c2cgeoform.schema.GeoFormSchemaNode(*args, **kw)¶

An SQLAlchemySchemaNode with deferred request and dbsession properties. This will allow defining schemas that requires the request and dbsession at module-scope.

Example usage:

schema = GeoFormSchemaNode(MyModel)

def create_form(request, dbsession):
    return Form(
        schema = schema.bind(
            request=request,
            dbsession=request.dbsession),
        ...
    )

add_unique_validator(column, column_id)¶

Adds an unique validator on this schema instance.

column: SQLAlchemy ColumnProperty that should be unique.
column_id: SQLAlchemy MapperProperty that is used to recognize the entity, basically the primary key ColumnProperty.

class c2cgeoform.schema.GeoFormManyToManySchemaNode(class_, includes=None, *args, **kw)¶

A GeoFormSchemaNode that properly handles many to many relationships.

includes:: Default to primary key name(s) only.

objectify(dict_, context=None)¶: Method override that returns the existing ORM class instance instead of creating a new one.

Configure the widgets¶

All Deform widgets can be used with c2cgeoform. See the Deform examples and widgets API reference for detailed description about available options.

Additionally, c2cgeoform provides some extra widgets:

class c2cgeoform.ext.deform_ext.FileUploadWidget(tmpstore, get_url=None, **kw)¶

Extension of deform.widget.FileUploadWidget to be used in a model class that extends the models.FileData mixin class.

Note that, contrary to deform.widget.FileUploadWidget, this extension is not meant to be used with the deform.FileData Colander type. Instead it works with the colander.Mapping type, which is what colanderalchemy uses for an SQLAlchemy model class.

Note also that it is required to set unknown to 'preserve' in the __colanderalchemy_config__ dictionary.

Example usage

from c2cgeoform import models
from c2cgeoform.ext import deform_ext

class Photo(models.FileData, Base):
    __tablename__ = 'photo'
    __colanderalchemy_config__ = {
        'title': _('Photo'),
        'unknown': 'preserve',
        'widget': deform_ext.FileUploadWidget(file_upload_temp_store)
    }
    permission_id = Column(Integer, ForeignKey('excavations.id'))

Attributes/Arguments

get_url (optional)

A callback function function(request, id) -> string which returns the URL to get the file. Example usage:

'widget': deform_ext.FileUploadWidget(
    _file_upload_temp_store,
    get_url=lambda request, id: request.route_url('file', id=id)
)

class c2cgeoform.ext.deform_ext.MapWidget(**kw)¶

A Deform widget that fits with GeoAlchemy 2 geometry columns and shows an OpenLayers 3 map which allows to draw and modify geometries.

Example usage

geom = Column(
    geoalchemy2.Geometry('POLYGON', 4326, management=True), info={
        'colanderalchemy': {
            'typ': colander_ext.Geometry(
                'POLYGON', srid=4326, map_srid=3857),
            'widget': deform_ext.MapWidget(
                base_layer='new ol.layer.Tile({ source: new ol.source.OSM() })',
                center=[829170, 5933942],
                zoom=7,
                fit_max_zoom=14
            )
        }})

To customize the map, the template file map.pt has to be overwritten.

Attributes/Arguments

base_layer (str, optional):: Javascript code returning the map base layer.
center ([x, y], optional):: Initial center when no geometry is given.
zoom (int, optional):: Initial zoom when no geometry is given.
fit_max_zoom (int, optional):: Maximum zoom when fitting to given geometry.

class c2cgeoform.ext.deform_ext.RecaptchaWidget(**kw)¶

A Deform widget for Google reCaptcha.

In c2cgeoform this widget can be used by setting the show_captcha flag when calling register_schema().

Example usage:

register_schema(
    'comment', model.Comment, show_confirmation=False,
    show_captcha=True,
    recaptcha_public_key=settings.get('recaptcha_public_key'),
    recaptcha_private_key=settings.get('recaptcha_private_key'))

Attributes/arguments

public_key (required): The Google reCaptcha site key.
private_key (required): The Google reCaptcha secret key.

class c2cgeoform.ext.deform_ext.RelationCheckBoxListWidget(model, id_field='id', label_field='label', order_by=None, **kw)¶

Extension of the widget ``deform.widget.CheckboxChoiceWidget which loads the values from the database using a SQLAlchemy model.

For n:m relations the widget can be used like so:

situations = relationship(
    "Situation",
    secondary=situation_for_permission,
    cascade="save-update,merge,refresh-expire",
    info={
        'colanderalchemy': {
            'title': _('Situations'),
            'widget': RelationCheckBoxListWidget(
                Situation,
                'id',
                'name',
                order_by='name',
                edit_url=lambda request, value: request.route_url(
                        'c2cgeoform_item',
                        table='situations',
                        id=value
                        )
            ),
            'includes': ['id'],
            'validator': manytomany_validator
        }
    })

Attributes/Arguments

model (required): The SQLAlchemy model that is used to generate the list of values.
id_field: The property of the model that is used as value. Default: id.
label_field: The property of the model that is used as label. Default: label.
order_by: The property of the model that is used for the order_by clause of the SQL query. Default: None.
edit_url (optionnal): a function taking request and value as parameter and returning an url to the correponding resource.

For further attributes, please refer to the documentation of deform.widget.Select2Widget in the deform documentation: <http://deform.readthedocs.org/en/latest/api.html>

class c2cgeoform.ext.deform_ext.RelationRadioChoiceWidget(model, id_field='id', label_field='label', order_by=None, **kw)¶

Extension of the widget ``deform.widget.RadioChoiceWidget which loads the values from the database using a SQLAlchemy model.

Example usage

districtId = Column(Integer, ForeignKey('district.id'), info={
    'colanderalchemy': {
        'title': 'District',
        'widget': deform_ext.RelationRadioChoiceWidget(
            District,
            'id',
            'name',
            order_by='name'
        )
    }})

The values of this field are filled with entries of model District, whereas property id is used as value and name as label.

Attributes/Arguments

model (required): The SQLAlchemy model that is used to generate the list of values.
id_field: The property of the model that is used as value. Default: id.
label_field: The property of the model that is used as label. Default: label.
order_by: The property of the model that is used for the order_by clause of the SQL query. Default: None.

For further attributes, please refer to the documentation of deform.widget.RadioChoiceWidget in the deform documentation: <http://deform.readthedocs.org/en/latest/api.html>

class c2cgeoform.ext.deform_ext.RelationSearchWidget(url, **kw)¶

A Deform widget to select an item via a search field. This widget is similar to the RelationSelectWidget, but instead of a select-box a Twitter Typeahead search field is shown.

Example usage:

address_id = Column(Integer, ForeignKey('address.id'), info={
    'colanderalchemy': {
        'title': _('Address'),
        'widget': deform_ext.RelationSearchWidget(
            url=lambda request: request.route_url('addresses'),
            model=Address,
            min_length=1,
            id_field='id',
            label_field='label'
        )
    }})

The user is responsible for providing a web-service at the given URL. The web service should expect requests of the form ?term=<search_terms>. And it should return responses of this form:

[{"id": 0, "label": "foo"}, {"id": 1, "label": "bar"}]

The name of the id and label keys are configurable. See below.

Attributes/arguments

url (required): The search service URL, or a function that takes a request a return the search service URL.
model (required): The SQLAlchemy model class associated to the linked table.
min_length: The minimum character length needed before suggestions start getting rendered. Default: 1.
id_field: The name of the “id” property in JSON responses. Default: "id".
label_field: The name of the “label” property in JSON responses. Default: "label".
limit: The maximum number of suggestions. Default: 8.

class c2cgeoform.ext.deform_ext.RelationSelect2Widget(model, id_field='id', label_field='label', default_value=None, order_by=None, **kw)¶

Extension of the widget ``deform.widget.Select2Widget which loads the values from the database using a SQLAlchemy model.

Example usage

districtId = Column(Integer, ForeignKey('district.id'), info={
    'colanderalchemy': {
        'title': 'District',
        'widget': deform_ext.RelationSelect2Widget(
            District,
            'id',
            'name',
            order_by='name',
            default_value=('', _('- Select -'))
        )
    }})

The values of this <select> field are filled with entries of model District, whereas property id is used as value and name as label.

For n:m relations the widget can be used like so:

situations = relationship(
    "Situation",
    secondary=situation_for_permission,
    cascade="save-update,merge,refresh-expire",
    info={
        'colanderalchemy': {
            'title': _('Situations'),
            'widget': RelationSelect2Widget(
                Situation,
                'id',
                'name',
                order_by='name',
                multiple=True
            ),
            'includes': ['id'],
            'validator': manytomany_validator
        }
    })

Attributes/Arguments

model (required): The SQLAlchemy model that is used to generate the list of values.
id_field: The property of the model that is used as value. Default: id.
label_field: The property of the model that is used as label. Default: label.
order_by: The property of the model that is used for the order_by clause of the SQL query. Default: None.
default_value: A default value that is added add the beginning of the list of values that were loaded from the database. For example: default_value=('', _('- Select -')) Default: None.
multiple: Allow to select multiple values. Requires a n:m relationship. Default: False.

For further attributes, please refer to the documentation of deform.widget.Select2Widget in the deform documentation: <http://deform.readthedocs.org/en/latest/api.html>

class c2cgeoform.ext.deform_ext.RelationSelectMapWidget(url, label_field='label', **kw)¶

A Deform widget to select an item on a map. From the idea, this widget is similar to the RelationSelectWidget, but instead of a select-box a map is shown.

Example usage

bus_stop = Column(Integer, ForeignKey('bus_stops.id'), info={
    'colanderalchemy': {
        'title': 'Bus stop',
        'widget': deform_ext.RelationSelectMapWidget(
            label_field='name', url='/bus_stops'
        )
    }})

The user is responsible for providing a web-service under the given URL, which returns a list of features as GeoJSON. The features must contain the two properties specified with id_field and label_field. The geometries are expected to use the CRS EPSG:4326.

To customize the map, the template file map_select.pt has to be overwritten.

Attributes/Arguments

url (required)

The URL to the web-service which returns the GeoJSON features or a callback function function(request) -> string which returns the URL to the web-service. Example usage:

'widget': deform_ext.RelationSelectMapWidget(
    label_field='name',
    url=lambda request: request.route_url('bus_stops')
)

label_field

The property of the GeoJSON features that is used as label. Default: label.

class c2cgeoform.ext.deform_ext.RelationSelectWidget(model, id_field='id', label_field='label', default_value=None, order_by=None, **kw)¶

Extension of the widget ``deform.widget.SelectWidget which loads the values from the database using a SQLAlchemy model.

Example usage

districtId = Column(Integer, ForeignKey('district.id'), info={
    'colanderalchemy': {
        'title': 'District',
        'widget': deform_ext.RelationSelectWidget(
            District,
            'id',
            'name',
            order_by='name',
            default_value=('', _('- Select -'))
        )
    }})

The values of this <select> field are filled with entries of model District, whereas property id is used as value and name as label.

For n:m relations the widget can be used like so:

situations = relationship(
    "Situation",
    secondary=situation_for_permission,
    cascade="save-update,merge,refresh-expire",
    info={
        'colanderalchemy': {
            'title': _('Situations'),
            'widget': RelationSelectWidget(
                Situation,
                'id',
                'name',
                order_by='name',
                multiple=True
            ),
            'includes': ['id'],
            'validator': manytomany_validator
        }
    })

Attributes/Arguments

model (required): The SQLAlchemy model that is used to generate the list of values.
id_field: The property of the model that is used as value. Default: id.
label_field: The property of the model that is used as label. Default: label.
order_by: The property of the model that is used for the order_by clause of the SQL query. Default: None.
default_value: A default value that is added add the beginning of the list of values that were loaded from the database. For example: default_value=('', _('- Select -')) Default: None.
multiple: Allow to select multiple values. Requires a n:m relationship. Default: False.

For further attributes, please refer to the documentation of deform.widget.SelectWidget in the deform documentation: <http://deform.readthedocs.org/en/latest/api.html>

Using custom templates¶

c2cgeoform distinguishes two types of templates: views templates and widget templates. - Views templates are used directly by Pyramid and provide the site structure. - Widgets templates are used by Deform to render the forms.

Default views templates¶

The default c2cgeoform views templates are located in the templates folder and use jinja2 syntax.

c2cgeoform comes with partial templates that are included in views templates of your project.

Overriding widgets templates globally¶

Deform widget templates are located in the templates/widgets folder and use the chameleon syntax.

At rendering time, Deform will search folders for the templates in order they appear in Form renderer search_path property. c2cgeoform configure it to:

default_search_paths = (
    resource_filename('c2cgeoform', 'templates/widgets'),
    resource_filename('deform', 'templates'))

But you can add you own widgets folder, in your package __init__.py file before including c2cgeoform using:

import c2cgeoform
search_paths = (
    (resource_filename(__name__, 'templates/widgets'),) +
    c2cgeoform.default_search_paths
)
c2cgeoform.default_search_paths = search_paths

To overwrite globally the Deform templates or the templates coming from c2cgeoform (like the map widget), you just need to copy the template to your application templates/widgets folder.

Use a custom template for a form or a specific widget in a form¶

Both the form main template and widget templates can be changed locally for a given model by giving a template property to the Widget.

base_schema = GeoFormSchemaNode(
    Comment,
    widget=FormWidget(template='comment'))

Note that it is possible to create a layout for the form fields without completely overriding the form template by giving a fields_template to the form schema.

base_schema = GeoFormSchemaNode(
    Comment,
    widget=FormWidget(fields_template='comment_fields'))

Here is the default one: https://github.com/camptocamp/c2cgeoform/blob/master/c2cgeoform/templates/widgets/mapping_fields.pt

Writing tests¶

Internationalization¶

Developer guide¶

This page describes how to set up the development environment for working on c2cgeoform. It is for developers working on c2cgeoform itself, not for developers working on c2cgeoform-based applications.

Note that c2cgeoform is a framework with a Pyramid scaffold used to create c2cgeoform-based applications. This scaffold produce a fully functional c2cgeoform-base project: the c2cgeoform_demo project.

When running code checks and tests, these jobs are first run on the c2cgeoform framework itself. Then the c2cgeoform_demo project is generated in .build folder. Finally, the checks and tests are launched in this project.

Note that you should never alter the c2cgeoform_demo project itself but the c2cgeoform scaffold and regenerate the c2cgeoform_demo project.

Clone the project¶

git clone git@github.com:camptocamp/c2cgeoform.git
cd c2cgeoform

Run the checks¶

make check

Run the tests¶

Create the tests database:

sudo -u postgres psql -c "CREATE USER \"www-data\" WITH PASSWORD 'www-data';"

export DATABASE=c2cgeoform_demo_tests
sudo -u postgres psql -d postgres -c "CREATE DATABASE $DATABASE OWNER \"www-data\";"
sudo -u postgres psql -d $DATABASE -c "CREATE EXTENSION postgis;"

Run the framework and demo tests:

make test

Serve the c2cgeoform_demo project¶

You need to create a PostGIS database. For example:

export DATABASE=c2cgeoform_demo
sudo -u postgres psql -d postgres -c "CREATE DATABASE $DATABASE OWNER \"www-data\";"
sudo -u postgres psql -d $DATABASE -c "CREATE EXTENSION postgis;"
make initdb

Run the development server:

make serve

You can now open the demo project in your favorite browser: http://localhost:6543/

And there you go, you’re ready to develop, make changes in c2cgeoform, run checks and tests in c2cgeoform. And finally see the results in c2cgeoform demo application.

Deploy the c2cgeoform_demo on demo server¶

Prepare the demo project:

# open a ssh connection with the GMF 2.3 server
ssh -A geomapfish-demo.camptocamp.com

# clone the c2cgeoform repository
cd /var/www/vhosts/geomapfish-demo/private
git clone git@github.com:camptocamp/c2cgeoform.git

# generate the c2cgeoform_demo project with mod_wsgi related files
APACHE_ENTRY_POINT=c2cgeoform make modwsgi

Create the database as to serve the development version, see: Serve the c2cgeoform_demo project

Include the demo project in Apache virtual host configuration:

echo "IncludeOptional $PWD/.build/c2cgeoform_demo/.build/apache.conf" > /var/www/vhosts/geomapfish-demo/conf/c2cgeoform_demo.conf
sudo apache2ctl configtest

If everything goes fine, restart apache:

sudo apache2ctl graceful

You can now open the demo project in your favorite browser: https://geomapfish-demo.camptocamp.com/c2cgeoform/

Welcome to c2cgeoform’s documentation!¶

Prerequisites¶

User guide¶

Creating a c2cgeoform project¶

Install c2cgeoform¶

Create a Pyramid project using c2cgeoform scaffold¶

Initialize a git repository¶

Install the project and its dependencies¶

Set up database¶

Run the development server¶

Defining the model for a form¶

Create the views for your model¶

Configure the grid¶

Understanding the schemas¶

Configure the widgets¶

Using custom templates¶

Default views templates¶

Overriding widgets templates globally¶

Use a custom template for a form or a specific widget in a form¶

Writing tests¶

Internationalization¶

Developer guide¶

Clone the project¶

Run the checks¶

Run the tests¶

Serve the c2cgeoform_demo project¶

Deploy the c2cgeoform_demo on demo server¶

Indices and tables¶