Stefano Apostolico's Blog - tech

Using multiple serializers with DjangoRestFramework

2017-10-10T00:00:00+02:00

How to use different serializers with DjangoRestFramework

Imagine to have a viewset and you want to parametrize the output, using different serializers.

class Person(models.Model):
    title = models.CharField(...)
    last_name = models.CharField(...)
    first_name = models.CharField(...)
    mail = models.EmailField(...)
    address = models.CharField(...)
    birthday = models.DateField()

class PersonSerializer(HouseSerializer):    
    full_name = serializers.SerializerMethodField()

    class Meta:
        model = House
        fields = ('title', 'last_name', 'first_name', 'mail', 'full_name')

    def get_full_name(self, obj):
        return "{0.last_name}, {1.first_name}".format(obj)

class PersonView(DynamicSerializerMixin, generics.ListAPIView):
    queryset = House.objects.all()
    serializer_class = HouseSerializer

it is easy to build a "serializer factory" to build on the fly

cache = {}

def serializer_factory(model, base=ModelSerializer, fields=None, exclude=None):
    attrs = {'model': model}
    if fields is not None:
        attrs['fields'] = fields
    if exclude is not None:
        attrs['exclude'] = exclude

    parent = (object,)
    if hasattr(base, 'Meta'):
        parent = (base.Meta, object)
    Meta = type(str('Meta'), parent, attrs)
    if model:
        class_name = model.__name__ + 'Serializer'
    else:
        class_name = 'Serializer'
    return type(base)(class_name, (base,), {'Meta': Meta, })


class DynamicSerializerMixin(object):
    serializers_fieldsets = {'std': None}
    serializer_class = ModelSerializer

    def get_serializer_class(self):
        ser = self.request.QUERY_PARAMS.get('serializer', 'std')
        if ser not in cache:                
            fields = self.serializers_fieldsets.get(ser, 'std')
            cache[ser] =  serializer_factory(self.model,
                                  self.serializer_class,
                                  fields=fields)

        return cache[ser]

adopt it:

class PersonView(DynamicSerializerMixin, generics.ListAPIView):
    queryset = Person.objects.all()
    serializer_class = PersonSerializer
    serializers_fieldsets = {'std': PersonSerializer,
                             'simple': ('first_name', 'last_name')
                             }

invoke it:

?serializer=std
?serializer=simpleA

py.test and Django

2016-11-10T00:00:00+01:00

Test django application with py.test and WebTest

I quit using default Django test suite some years ago, sometimes I go back to it to see if something new can change my mind, but up to now I continue to prefer my choice:

py.test
WebTest with django-webtest.

Why?

Several reasons - here's a few:

It can run unittest, doctest, and nose, style tests suites, making it ideal for new and legacy projects.
It includes an intuitively named raises context manager for testing exceptions.
You can define fixture arguments to generate baseline data. This is very, very different from Django-style fixtures.
Via pytest.ini/setup.cfg you can change the behavior of pytest.
Integrates nicely with setup.py.
WebTest allows you to model a user's experience much more closely as it is smart about mark-up. Instead of hand-crafting GET and POST requests, you can use the WebTest API to follow links and submit forms - this is what users actually do. As a result, your tests accurately capture user stories.
Writing functional tests with WebTest is both easier and quicker than using Django's test client. It's much simpler to fill in forms that construct complicated arrays of POST data - this is particularly noticable with formsets.
The WebTest response object supports several ways of parsing the response HTML, making it easy to make complicated assertions about the response.
response.showbrowser() is simply amazing

False Myths: Customers does not care about technology they care about money

2016-10-26T00:00:00+02:00

Yesterday a sentence like this was posted in a collaboration platform:

"At the end of the day your customers don’t care whether you’re agile, lean or practice design thinking."

I fully agree with the essence of this idea, but it does sounds misleading in one aspect of presentation: This sentence, when interpreted as "customers only care about money", could be a huge mistake.

Just because our customers do not care how we work, it does not mean that we should not care about methodologies and process. I don't care what products and tools my dentist uses; it's upto him, but I expect he cares about them a lot! Tools, processes and metodologies are in our domain and scope - though they do not need to be endorsed by or exposed to our customers, without an internal professional approach, we cannot deliver great products and services that solve meaningful problems for them in effective ways.

When the Business/Customer allocates a budget, there's an expectation of return value: a working, reliable product delivered in time and within budget. If something goes wrong, the Business/Customer complains about money because it is the only "unit of measure" that he/she knows that is understandable in our domain, and is the only common language he/she can speak with us.

As an analogy, When a new-bought car breaks down after a week, we go back to the vendor and complain

"Hey, I paid $$$ for this car but it does not work anymore!",

only because we do not know how to say

"Hey, this car has a design error , you should not have sold it!".

Ultimately, businesses care about "good products", but the way to deliver them cannot pass through a wrong approach or ignoring any methodology only because businesses don't care about them "explicitly".

Again, as an analogy from our experience on flights, I don't think we would expect our planes to be driven overlooking the current best practices, procedures and commonsense, or, even worst, piloted by the first available person.

Skills, knowledge and mastering the domain is the only way to give to the Business what it expects, any another approach is only a workaroud that undermines our reputation and credibility day after day.

About rules, best practices and experience

2015-10-07T00:00:00+02:00

What does distinguishes an architect by a good architect, a developer from a good developer ?

Sometimes customers ask me a list of rules or best practices and guidelines to adopt in any architecture and/or application.

Every time, I try to explain that this list does not exist, is like create a list of ingredients valid for all the recipes

If you ask any developer 5 rules to follow to have the perfect code he will look at you astonished and reply that there are many "rules" but each time the important one is different.

But if you show a badly written code, good programmer will find the issue and will be able to explain you why, sometimes simply annotating them as "code smell"

A good architect is not who apply all the patterns and best practices (virtually impossible), a good architect chooses which pattern can ignore and which "best practices" can be violated

Design has nothing related to rules and laws, is judgement, skill, knowledge, experience and high capacity for abstraction

PostGIS, PostgreSQL, and Django on OS X

2015-06-06T00:00:00+02:00

Here's how I got through the installation and configuration of PostGIS on OS X (10.10 Yosemite).

This post is mainly for my reference. I spent too much time to find a clear reproducible way to achieve the result.

Here's what finally worked for me.

The stack

Postgresql 9.4.3
PostGIS 2.1.7
Python 2.7
Django 1.7

Install PostGIS

I'm not sure I could have actually accomplished this without Homebrew. It really is a great package management tool for OS X. It handles dependencies with no pain at all.

get package info

brew info postgis

this produce something like (partial):

postgis: stable 2.1.7, HEAD
Adds support for geographic objects to PostgreSQL
http://postgis.net
Not installed

Install PostGIS

If the version you got does not match this post, try with:

brew tap postgis
brew search postgis
brew install homebrew/versions/postgis20

otherwise:

brew install postgis

Homebrew will install both PostGIS and PostgreSQL, check it with

brew info postgres
brew info postgis

the default data dir /usr/local/var/postgres is owned by root and I prefre to run my database as non privileged user, first I need to chown the directory

sudo chown sax /usr/local/var/postgres

Initialize

initdb /usr/local/var/postgres  -E utf8

Launch postgres

pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/server.log start

To do not receive annoying messages during my daily activities I also create the standard postgres superuser

createuser -s -r postgres

PostGIS Functions and Template

PostGIS essentially is a table a couple of databse views and a many functions. You need to create a "template" database to store all of them.

Create the database that we use as template for any

createdb template_postgis
psql -f /usr/local/share/postgis/postgis.sql template_postgis
psql -f /usr/local/share/postgis/spatial_ref_sys.sql template_postgis

Create Django database

psql -c 'CREATE DATABASE wfp_commonapi_gis;' -U postgres;
psql -d gis_example -c 'CREATE EXTENSION postgis;' -U postgres;

createdb -E UTF8 PIPPO -T template_postgis

NOTE: Do not run django test before you create the "real" PostGIS database (at least with pytest) or you get a OperationalError: FATAL: database "DBNAME" does not exist. Seems that Django is not able to create a PostGIS test database if the original one does not exists.

Make PostGIS start automatically

This is done following the hint on bottom of brew info postgres output

ln -sfv /usr/local/opt/postgresql/*.plist ~/Library/LaunchAgents 
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.postgresql.plist

Put everything togheter

#!/usr/bin/env bash

PGDATA="/usr/local/var/postgres"
LOG="/usr/local/var/postgres/server.log"

brew install postgis || exit 1
sudo chown sax ${PGDATA}  || exit 1
initdb ${PGDATA} -E utf8  || exit 1

pg_ctl -D ${PGDATA} -l ${LOG} start -w || exit 1

createuser -s -r postgres  || exit 1

createdb template_postgis || exit 1

psql -f /usr/local/share/postgis/postgis.sql template_postgis  || exit 1
psql -f /usr/local/share/postgis/spatial_ref_sys.sql template_postgis  || exit 1

pg_ctl status