Introduction

WebODM is a free, user-friendly, extendable application and API for drone image processing. It generates georeferenced maps, point clouds and textured 3D models from aerial images.

Developers can leverage this API to extend the functionality of WebODM or integrate it with existing software like QGIS or AutoCAD.

Quickstart

How To Process Images

In this tutorial we’ll explore how to process an orthophoto from a set of aerial images using Python. To do that we’ll need to:

• Authenticate
• Create a Project. Projects are a way to group together related Task items
• Upload some images to create a Task
• Check for Task progress. Photogrammetry can take a long time, so results could take a few minutes to a few hours to be processed.
• Download the resulting orthophoto.

import requests
res = requests.post('http://localhost:8000/api/token-auth/', 
                    data={'username': 'admin',
                          'password': 'admin'}).json()
token = res['token']

First, we authenticate with WebODM. A token is returned when authentication is successful.

res = requests.post('http://localhost:8000/api/projects/', 
                    headers={'Authorization': 'JWT {}'.format(token)},
                    data={'name': 'Hello WebODM!'}).json()
project_id = res['id']

Then we need to create a Project. We pass our token via the Authorization header. If we forget to pass this header, the system will not authenticate us and will refuse to process the request. We also assign a name to our project.

images = [
    ('images', ('image1.jpg', open('image1.jpg', 'rb'), 'image/jpg')), 
    ('images', ('image2.jpg', open('image2.jpg', 'rb'), 'image/jpg')),
    # ...
]
options = json.dumps([
    {'name': "orthophoto-resolution", 'value': 24}
])

res = requests.post('http://localhost:8000/api/projects/{}/tasks/'.format(project_id), 
            headers={'Authorization': 'JWT {}'.format(token)},
            files=images,
            data={
                'options': options
            }).json()

task_id = res['id']

We can then create a Task. The only required parameter is a list of multiple, multipart-encoded images. Processing will start automatically as soon as a Processing Node is available. It is possible to specify additional options by passing an options value, which is a JSON-encoded list of name/value pairs. Several other options are available. See the Task - Processing Options reference for more information.

while True:
    res = requests.get('http://localhost:8000/api/projects/{}/tasks/{}/'.format(project_id, task_id), 
                headers={'Authorization': 'JWT {}'.format(token)}).json()

    if res['status'] == status_codes.COMPLETED:
        print("Task has completed!")
        break
    elif res['status'] == status_codes.FAILED:
        print("Task failed: {}".format(res))
        sys.exit(1)
    else:
        print("Processing, hold on...")
        time.sleep(3)

We periodically check for the Task status using a loop.

res = requests.get("http://localhost:8000/api/projects/{}/tasks/{}/download/orthophoto.tif".format(project_id, task_id), 
                        headers={'Authorization': 'JWT {}'.format(token)},
                        stream=True)
with open("orthophoto.tif", 'wb') as f:
    for chunk in res.iter_content(chunk_size=1024): 
        if chunk:
            f.write(chunk)
print("Saved ./orthophoto.tif")

Our orthophoto is ready to be downloaded. A variety of other assets, including a dense 3D point cloud and a textured model are also available.

Congratulations! You just processed some images.

Reference

Authentication

Authentication Basics

Get authentication token:

curl -X POST -d "username=testuser&password=testpass" http://localhost:8000/api/token-auth/

{"token":"eyJ0eXAiO..."}

Use authentication token:

curl -H "Authorization: JWT <your_token>" http://localhost:8000/api/projects/

{"count":13, ...}

Use authentication token via querystring (less secure):

curl http://localhost:8000/api/projects/?jwt=<your_token>

{"count":13, ...}

POST /api/token-auth/

To access the API, you need to provide a valid username and password. You can create users from WebODM’s Administration page.

If authentication is successful, you will be issued a token. All API calls should include the following header:

The token expires after a set amount of time. See Token Expiration for more information.

Since applications sometimes do not allow headers to be modified, you can also authenticate by appending the jwt querystring parameter to a protected URL. This is less secure, so pass the token via header if possible.

Token Expiration

The token expires after six hours by default. The expiration time is defined in the settings module of Django in WebODM. If building WebODM from sources or running it natively, the expiration time can be changed in the JWT_AUTH['JWT_EXPIRATION_DELTA'] variable. Otherwise, e.g. using the docker images, you will have to request another token when a token expires.

You know that a token has expired if any API call returns a 403 status code with the JSON body {'detail': 'Signature has expired.'}.

Project

Example project:

{
    "id": 2,
    "tasks": [
        7,
        6,
        5
    ],
    "created_at": "2016-12-07T02:09:28.515319Z",
    "name": "Test",
    "description": "",
    "permissions": [
        "delete",
        "change",
        "add",
        "view"
    ]
}

A Project is a collection of Task items.

Create a project

POST /api/projects/

Update a project

PATCH /api/projects/{id}/

Parameters are the same as above.

Delete a project

DELETE /api/projects/{id}/

Upon deletion, all Task items associated with the Project are deleted also. The operation is irreversible.

Get single project

GET /api/projects/{id}/

Get list of projects

Project list:

{
    "count": 1,
    "next": null,
    "previous": null,
    "results": [
        {
            "id": 2,
            "tasks": [
                7,
                6,
                5
            ],
            "created_at": "2016-12-07T02:09:28.515319Z",
            "name": "Test",
            "description": ""
        }
    ]
}

GET /api/projects/

Example: Filtering by name

GET /api/projects/?name=hello

Retrieves projects that have a name of “hello”.

Example: Sorting

GET /api/projects/?ordering=-id

Sort by project ID, descending order.

Task

Example task:

{
  "id": 134,
  "project": 27,
  "processing_node": 10,
  "processing_node_name": "localhost:3000",
  "images_count": 48,
  "can_rerun_from": [],
  "available_assets": [
    "all.zip",
    "orthophoto.tif",
    "orthophoto.png",
    "georeferenced_model.las",
    "georeferenced_model.ply",
    "georeferenced_model.csv",
    "textured_model.zip"
  ],
  "uuid": "4338d684-91b4-49a2-b907-8ba171894393",
  "name": "Task Name",
  "processing_time": 2197417,
  "auto_processing_node": false,
  "status": 40,
  "last_error": null,
  "options": [
    {
      "name": "use-opensfm-pointcloud",
      "value": true
    }
  ],
  "created_at": "2017-02-18T18:01:55.402551Z",
  "pending_action": null,
  "upload_progress": 1.0,
  "resize_progress": 0.0,
  "running_progress": 1.0
}

A Task is the basic processing unit of WebODM. To compute an orthophoto, point cloud and textured model from a set of images, you need to create a Task.

Create a task

POST /api/projects/{project_id}/tasks/

You assign a Task to a Project by passing the proper project_id path in the URL.

Update a task

PATCH /api/projects/{project_id}/tasks/{task_id}/

Parameters are the same as above.

Import Task

POST /api/projects/{project_id}/tasks/import

Import task that have been processed by another WebODM instance (or via webodm.net or NodeODM)

You have to provide either filename or url parameter (but not both) in order to import created assets.

Remember to set proper Content-type for the request depending on how the assets are uploaded:

Get list of tasks

Task list:

[
    {
        "id": 6,
        "project": 2,
        "processing_node": 2,
        "processing_node_name": "localhost:3000",
        "images_count": 89,
        "uuid": "2e8b687d-c269-4e2f-91b3-5a2cd51b5321",
        "name": "Test name",
        "processing_time": 8402184,
        "auto_processing_node": true,
        "status": 40,
        "last_error": null,
        "options": [],
        "created_at": "2016-12-08T13:32:28.139474Z",
        "pending_action": null,
        "upload_progress": 1.0,
        "resize_progress": 0.0,
        "running_progress": 1.0
    }
]

GET /api/projects/{project_id}/tasks/

Retrieves all Task items associated with project_id.

Download assets

GET /api/projects/{project_id}/tasks/{task_id}/download/{asset}

After a task has been successfully processed, the user can download several assets from this URL. Not all assets are always available. For example if GPS information is missing from the input images, the orthophoto.tif asset will be missing. You can check the available_assets property of a Task to see which assets are available for download.

Download assets (raw path)

GET /api/projects/{project_id}/tasks/{task_id}/assets/{path}

After a task has been successfully processed, its assets are stored in a directory on the file system. This API call allows direct access to the files in that directory (by default: WebODM/app/media/project/{project_id}/task/{task_id}/assets). This can be useful to those applications that want to stream a Potree dataset, or render a textured 3D model on the fly.

Retrieve console output

Console output example:

curl -H "Authorization: JWT <your_token>" http://localhost:8000/api/projects/2/tasks/1/output/?line=5

[DEBUG]   /var/www/data/e453747f-5fd4-4654-9622-b02727b29fc5/images\n[DEBUG]   Loaded DJI_0219.JPG | camera: dji fc300s ...

GET /api/projects/{project_id}/tasks/{task_id}/output/

As a Task is being processed, processing nodes will return an output string that can be used for debugging and informative purposes. Output is only available after processing has started.

Cancel task

POST /api/projects/{project_id}/tasks/{task_id}/cancel/

Stop processing a Task. Canceled tasks can be restarted.

Remove task

POST /api/projects/{project_id}/tasks/{task_id}/remove/

All assets associated with it will be destroyed also. If the Task is currently being processed, processing will stop.

Restart task

POST /api/projects/{project_id}/tasks/{task_id}/restart/

If a Task has been canceled or has failed processing, or has completed but the user decided to change processing options, it can be restarted. If the Processing Node assigned to the Task has not changed, processing will happen more quickly compared to creating a new Task, since the Processing Node remembers the uuid of the Task and will attempt to reuse previous results from the computation pipeline.

Orthophoto TMS layer

GET /api/projects/{project_id}/tasks/{task_id}/orthophoto/tiles.json

GET /api/projects/{project_id}/tasks/{task_id}/orthophoto/tiles/{Z}/{X}/{Y}.png

After a task has been successfully processed, a TMS layer is made available for inclusion in programs such as Leaflet or Cesium.

Surface Model TMS layer

GET /api/projects/{project_id}/tasks/{task_id}/dsm/tiles.json

GET /api/projects/{project_id}/tasks/{task_id}/dsm/tiles/{Z}/{X}/{Y}.png

Terrain Model TMS layer

GET /api/projects/{project_id}/tasks/{task_id}/dtm/tiles.json

GET /api/projects/{project_id}/tasks/{task_id}/dtm/tiles/{Z}/{X}/{Y}.png

Pending Actions

In some circumstances, a Task can have a pending action that requires some amount of time to be performed.

Status Codes

Processing Node

Example processing node:

{
    "id": 2,
    "online": true,
    "hostname": "nodeodm.masseranolabs.com",
    "port": 80,
    "api_version": "1.0.1",
    "engine_version": "0.6.0",
    "engine": "odm",
    "last_refreshed": "2017-03-01T21:14:49.918276Z",
    "queue_count": 0,
    "max_images": null,
    "label": "nodeodm.masseranolabs.com:80",
    "available_options": [
        {
            "help": "Oct-tree depth at which the Laplacian equation is solved in the surface reconstruction step. Increasing this value increases computation times slightly but helps reduce memory usage. Default: 9",
            "name": "mesh-solver-divide",
            "type": "int",
            "value": "9",
            "domain": "positive integer"
        },
    ...

Processing nodes are associated with zero or more tasks and take care of processing input images. Processing nodes are computers or virtual machines running NodeODM or any other API compatible with it.

Available Options

Add a processing node

POST /api/processingnodes/

All other fields are automatically populated, and shouldn’t generally be specified.

Update a processing node

PATCH /api/processingnodes/

Parameters are the same as above.

Delete a processing node

DELETE /api/processingnodes/

Upon deletion, all Task items associated with the node will continue to exist. You might get errors (duh!) if you delete a processing node in the middle of processing a Task.

Get list of processing nodes

GET /api/processingnodes/

Example: Show only nodes that have a valid set of options

GET /api/processingnodes/?has_available_options=true

Example: Sorting

GET /api/processingnodes/?ordering=-hostname

Sort by hostname, descending order.

Processing Options

Processing options example:

[
    {
        "help": "Oct-tree depth at which the Laplacian equation is solved in the surface reconstruction step. Increasing this value increases computation times slightly but helps reduce memory usage. Default: 9",
        "name": "mesh-solver-divide",
        "type": "int",
        "value": "9",
        "domain": "positive integer"
    },
    {
        "help": "Ignore matched keypoints if the two images share less than <float> percent of keypoints. Default: 2",
        "name": "matcher-threshold",
        "type": "float",
        "value": "2",
        "domain": "percent"
    },
    ...

GET /api/processingnodes/options/

Display the common options available among all online processing nodes. This is calculated by intersecting the available_options field of all online processing nodes visible to the current user.

Use this list of options to check whether a particular option is supported by all online processing nodes. If you use the automatic processing node assignment feature for processing tasks, this is the list you want to display to the user for choosing the options to use during processing.

Permissions

WebODM comes with a standard model level permission system. You can check whether users are logged-in and have privileges to act on things model-wise (can a user add a project? can a user view projects?).

On top of that, WebODM features a powerful row level permission system. You can specify exactly which things a user has or has not access to, delete, change, etc.

Changes to the permissions of objects can be handled via the Administration page of WebODM.

We are planning to make it easier for users and developers to handle permissions via an API. This is a work in progress.

Permission Values

Admin/Users

Example User

{
    "id": 1,
    "password": "pbkdf2_sha256$120000$vkzUnKJwwaNl$95nqgBjqZ3/8Plk5soe2SjUPEF5fFNjBDfIapOXCy/Y=",
    "last_login": "2019-09-12T01:45:05Z",
    "is_superuser": true,
    "username": "admin",
    "first_name": "",
    "last_name": "",
    "email": "[email protected]",
    "is_staff": true,
    "is_active": true,
    "date_joined": "2019-09-12T01:44:18Z",
    "groups": [
        1
    ],
    "user_permissions": [
        9,
        10,
        11,
        12
    ]
}

This API can only be used by admin users.

Create a user

POST /api/admin/users/

Update a user

PUT /api/admin/users/{id}/

Parameters are the same as above.

Delete a user

DELETE /api/admin/users/{id}/

Get a user

GET /api/admin/users/{id}/

Get list of users

GET /api/admin/users/

Example: Filtering by email

GET /api/admin/users/[email protected]

Retrieves projects that have a email of “[email protected]”.

Admin/Groups

Example Group

{
    "id": 1,
    "name": "Xyz",
    "permissions": [
        53,
        54,
        55,
        56,
        37,
        38,
        39,
        40,
        49,
        50,
        51,
        52,
        76
    ]
}

This API can only be used by admin users.

Create a group

POST /api/admin/groups/

Update a group

PUT /api/admin/groups/{id}/

Parameters are the same as above.

Delete a group

DELETE /api/admin/groups/{id}/

Get a group

GET /api/admin/groups/{id}/

Get list of group

GET /api/admin/groups/

Example: Filtering by email

GET /api/admin/groups/?name=Xyz

Retrieves projects that have a name of “Xyz”.

Handling Errors

All API calls use the status codes as described in the Django REST Framework’s Status Code Guide, but generally you only need to check for success status codes (200 or 204), handle the special case of Token Expiration (403) and report an error otherwise.

Error Status Codes

This is not an exhaustive list, but common error codes are listed below.

For security reasons, sometimes an operation which should return 403 returns 404 to avoid disclosing IDs and other information to attackers.

For Developers

Development Quickstart

1. Make a fork of the WebODM repository
2. Clone your repository in a directory
3. Create a new branch: git checkout -b branchname.
4. Setup a development environment either with docker or natively.
5. Commit the changes: git commit -a -m "describe your changes"
6. Push the changes to your repository: git push origin branchname
7. Create a pull request

We don’t have many rules. Follow the guidelines indicated in the Contributing document, be nice to others and you’ll do great! :)

Setup a Development Environment

There are two ways to setup a development environment. The easiest one is to use docker.

Once you have a development environment, read about the project overview and get hacking!

Docker Setup

Follow the Getting Started instructions, then run:

./webodm.sh start --dev

That’s it! You can modify any of the files, including SASS and React.js files. Changes will be reflected in the running WebODM instance automatically.

Native Setup

If you can follow the instructions to run WebODM natively, you should be able to make changes to the code directly.

Run Unit Tests

We think testing is a necessary part of delivering robust software. We try to achieve complete test coverage for backend code and at a minimum robust smoke testing for frontend code.

To run the unit tests, simply type:

./webodm.sh test

Apply Changes In Production

Once you’re done making changes, if you start WebODM in production mode (without the --dev flag), you will notice that your changes are missing. This is because webodm.sh uses the opendronemap/webodm_webapp docker image to launch WebODM, which doesn’t have your changes. To apply the changes, you need to rebuild the docker image locally:

docker build -t opendronemap/webodm_webapp .

You can also modify the docker-compose.yml file to point to a different image.

Project Overview

Backend

The backend is based mainly on Django and Django REST Framework.

We don’t use much of Django’s templating system, except for the Administration and Processing Nodes sections. Instead we use Django to expose an API, which we then tie to a React.js app.

Directories of interest are listed as follow:

Frontend

We use a React.js app (ES6 syntax) and SCSS for various UI components such as the dashboard. We use webpack to build intermediate components into a static bundle.

Directories of interest are listed as follow:

/app/static/app/js/main.jsx is the entry point for the UI. If you wonder how we tie Django and React.js together, this is the file to look at to begin your search.

Documentation

We use Slate to generate our documentation. See their project’s wiki for information about making changes to the documentation.

Documentation can be changed by modifying the files in /slate/source/includes.