Total Open Station 0.4 release

This article was originally published on the Total Open Station website at https://tops.iosa.it/

After two years of slow development, I took the opportunity of some days off to finally release version 0.4, that was already available in beta since 2017.

No open bugs were left and this release is mature enough to hit the repositories.

Find it on PyPI at https://pypi.python.org/pypi/totalopenstation as usual.

Windows users, please note that the TOPS-on-a-USB-stick version will have to wait a few days more, but the beta version is equally functional.

What’s new in Total Open Station 0.4

The new version brings read support for 4 new formats:

  • Carlson RW5
  • Leica GSI
  • Sokkia SDR33
  • Zeiss R5

Other input formats were improved, most notably Nikon RAW.

DXF output was improved, even though the default template is not very useful since it is based on an old need from the time when TOPS was developed day to day on archaeological excavations.

The work behind these new formats is in part by the new contributor to the project, Damien Gaignon (find him as @psolyca on GitHub), who submitted a lot of other code and started helping with project maintenance as well. I am very happy to have Damien onboard and since my usage of TOPS is almost at zero, it’s very likely that I will hand over the development in the near future.

The internal data structures for handling the conversion between input and output formats are completely new, and based on the Python GeoInterface abstraction offered by the pygeoif library. This allows going beyond single points to managing lines and polygons, even though no such feature is available at the moment. If you often record linear or polygonal features that you’re manually joining in the post-processing stage, think about helping TOPS development and you could get DXF or Shapefiles with the geometries ready to use (yes, Shapefile output is on our plans, too).

There were many bugfixes, more than 100 commits, 64 by Damien Gaignon and 52 by myself (to be honest, many of my own commits are just merges!).

This version is the last built on Python 2, and work is already ongoing towards a new version that will be based on Python 3: a more mature codebase will mean a better program, without any visible drastic change.

Photo by Scott Blake on Unsplash

IOSACal 0.4

IOSACal is an open source program for calibration of radiocarbon dates.

A few days ago I released version 0.4, that can be installed from PyPI or from source. The documentation and website is at http://c14.iosa.it/ as usual. You will need to have Python 3 already installed.

The main highlight of this release are the new classes for summed probability distributions (SPD) and paleodemography, contributed by Mario Gutiérrez-Roig as part of his work for the PALEODEM project at IPHES.

A bug affecting calibrated date ranges extending to the present was corrected.

On the technical side the most notable changes are the following:

  • requires NumPy 1.14, SciPy 1.1 and Matplotlib 2.2
  • removed dependencies on obsolete functions
  • improved the command line interface

You can cite IOSACal in your work with the DOI https://doi.org/10.5281/zenodo.630455. This helps the author and contributors to get some recognition for creating and maintaining this software free for everyone.

What’s the correlation between the exposure time of your photographs and the time of the day?

My digital photo archive spans 15 years and holds about 12,600 pictures (not so many, after all). I’m curious to see if there is a correlation between the exposure time of my photographs and the time of the day they were taken. A rather simplistic observation, perhaps.

In short: there’s nothing spectacular about this correlation, but it’s nice. The morning hours are the ones with the lowest average exposure (the plot is reversed, so you can look at familiar integer numbers) time at around 1/320 s between 9 and 10 AM. There’s a sharp increase between 12 and 1 PM, then it increases again after 4 PM towards dusk. I don’t take many pictures at night!

See for yourself.

expotime2

The most frequent values for exposure time are in the table below. 1/30 s is the typical exposure time when using the flash, and it’s recognisable in the plot above.

1/n s occurrences
800 1986
1000 1178
30 943
400 547
250 488
640 458
200 450
500 388
320 342
160 337

The Python and R scripts are at https://gitlab.com/steko/expotime (giving GitLab a spin since GitHub is a monopoly and I don’t like that). I’m still doing some experiments with the source data, then I’ll upload those as well.

Archaeology and Django: mind your jargon

I have been writing small Django apps for archaeology since 2009 ‒ Django 1.0 had been released a few months earlier. I love Django as a programming framework: my initial choice was based on the ORM, at that time the only geo-enabled ORM that could be used out of the box, and years later GeoDjango still rocks. I almost immediately found out that the admin interface was a huge game-changer: instead of wasting weeks writing boilerplate CRUD, I could just focus on adding actual content and developing the frontend. Having your data model as source code (under version control) is the right thing for me and I cannot go back to using “database” programs like Access, FileMaker or LibreOffice.

Previous work with Django in archaeology

There is some prior art on using Django in the magic field of archaeology, this is what I got from published literature in the field of computer applications in archaeology:

I have been discussing this interaction with Diego Gnesi Bartolani for some time now and he is developing another Django app. Python programming skills are becoming more common among archaeologists and it is not surprising that databases big and small are moving away from desktop-based solutions to web-based

The ceramicist’s jargon

There is one big problem with Django as a tool for archaeological data management: language. Here are some words that are either Python reserved keywords or very important in Django:

  • class (Python keyword)
  • type (Python keyword)
  • object (Python keyword)
  • form (HTML element, Django module)
  • site (Django contrib app)

Unfortunately, these words are not only generic enough to be used in everyday speak, but they are very common in the archaeologist’s jargon, especially for ceramicists.

Class is often used to describe a generic and wide group of objects, e.g. “amphorae”, “fine ware”, “lamps”, ”cooking ware” are classes of ceramic products ‒ i.e. categories. Sometimes class is also used for narrower categories such as “terra sigillata italica”, but the most accepted term in that case is ware. The definition of ware is ambiguous, and it can be based on several different criteria: chemical/geological analysis of source material; visible characteristics such as paint, decoration, manufacturing; typology. The upside is that ware has no meaning in either Python or Django.

Form and type are both used within typologies. There are contrasting uses of these two terms:

  •  a form defines a broad category, tightly linked to function (e.g. dish, drinking cup, hydria, cythera) and a type defines a very specific instance of that form (e.g. Dragendorff 29); sub-types are allowed and this is in my experience the most widespread terminology;
  • a form is a specific instance of a broader function-based category ‒ this terminology is used by John W. Hayes in his Late Roman Pottery.

These terminology problems, regardless of their cause, are complicated by translation from one language to another, and regional/local traditions. Wikipedia has a short but useful description of the general issues of ceramic typology at the Type (archaeology) page.

Site is perhaps the best understood source of confusion, and the less problematic. First of all everyone knows that the word site can have a lots of meanings and lots of archaeologists survive using both the website and the archaeological site meaning everyday. Secondly, even though the sites app is included by default in Django, it is not so ubiquitous ‒ I always used it only when deploying, una tantum.

Object is a generic word. Shame on every programming language designer who ever thought it was a good idea to use such a generic word in a programming language, eventually polluting natural language in this digital age. No matter how strongly you think object is a good term to designate archaeological finds, items, artifacts, features, layers, deposits and so on, thou shalt not use object when creating database fields, programming functions, visualisation interfaces or anything else, really.

The horror is when you end up writing code like this:

class Class(models.Model)
    '''A class. Both a Python class and a classification category.'''

    pass

class Type(models.Model)
    '''A type. Actually, a Python class.

    >>> t = Type()
    >>> type(t)
    <class '__main__.Type'>
    '''

Not nice.

Is there a solution to this mess? Yes. As any serious Pythonista knows…

Explicit is better than implicit.
[…]
Namespaces are one honking great idea — let’s do more of those!

The Zen of Python

Since changing the Python syntax is not a great idea, the best solution is to prefix anything potentially ambiguous to make it explicit (as suggested by the honking idea of namespaces ‒ a prefix is a poor man’s namespace). If you follow this, or a likewise approach, you won’t be left wondering if that form is an HTML form or a category of ceramic items.

# pseudo-models.py

class CeramicClass(models.Model):
    '''A wide category of ceramic items, comprising many forms.'''

    name = models.CharField()

class CeramicForm(models.Model):
    '''A ceramic form. Totally different from CeramicType.'''

    name = models.CharField()

class CeramicType(models.Model):
    '''A ceramic type. Whatever that means.'''

    name = models.CharField()
    ceramic_class = models.ForeignKey(CeramicClass)
    ceramic_form = models.ForeignKey(CeramicForm)
    source_ref = models.URLField()

class ArcheoSite(models.Model):
    '''A friendly, muddy, rotting archaeological site.'''

    name = models.CharField()

class CeramicFind(models.Model):
    '''The real thing you can touch and look at.'''

    ceramic_type = models.ForeignKey(CeramicType)
    archeo_site = models.ForeignKey(ArcheoSite)
    ... # billions of other fields