Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Parsing fields in django-import-export before importing

I am using django-import-export package to expect a csv file containing a location's name and its longitude and latitude.

I want to parse the longitude and latitude field from the csv to convert them into django.contrib.gis.geos.Point object so that I can input it to my Location model's geom field.

# models.py
from django.contrib.gis.db import models
class Location(models.Model):
    name = models.CharField(max_length=200)
    geom = models.PointField(null=True, blank=True)

    def __str__(self):
        return self.name

# admin.py
from .models import Location
from import_export import resources
from import_export.admin import ImportExportModelAdmin

class LocationResource(resources.ModelResource):
    geom = Field()
    latitude = Field()
    longitude = Field()

    class Meta:
        model = Location
        fields = ('id','name', 'latitude', 'longitude')
        exclude = ('geom')
        export_order = ('id', 'name', 'latitude', 'longitude')

    def dehydrate_geom(self, data):
        return Point(data.longitude, data.longitude)

class LocationAdmin(ImportExportModelAdmin):
    resource_class = LocationResource

admin.site.register(Location, LocationAdmin)

This is how far I got but to no success. Must have:

Location(name='name', geom=Point(longitude, latitude))

CSV file: locations.csv

id,name,longitude,latitude
1,Naga,120.18,18.20

UPDATE 1 Tried using hydrate_<field_name> but with no success.

class ProjectActivityResource(resources.ModelResource):
    latitude = Field(attribute='latitude', column_name='latitude')
    longitude = Field(attribute='longitude', column_name='longitude')

    class Meta:
        model = ProjectActivity
        fields = ('id', 'project_site', 'name', 'latitude',
                        'longitude', 'date_updated')
        exclude = ('geom')
        export_order = ('id', 'project_site', 'name', 'latitude',
                        'longitude', 'date_updated')

    def hydrate_geom(self, project_activity):
        print(project_activity)
        return Point(float(project_activity.longitude), float(project_activity.longitude))
like image 264
Nikko Avatar asked Jul 27 '19 14:07

Nikko


People also ask

What are imports in Python and Django?

Imports are an inevitable part of Python and Django development. Pep8, which is the official style guide for Python, recommends imports be placed at the top of the file, on separate lines, and grouped in the following order: Whenever possible, be as explicit as possible with imports.

What are absolute imports in Django?

The top 3 lines are absolute imports which are used when importing packages from outside a given app. This is how all Django core code is imported. The database model is imported using an explicit relative import --we didn't hardcode the app name in here which makes it much more reusable.

What is skip row in Django import export?

Skipping rows is a powerful import-export method that helps in skipping rows in certain violations of data validations. For eg — If you have a large number of User Profile data that is needed to import but these data contain duplicate profiles also. You can override skip_row method to avoid Django’s unique data integrity.

What is Django-import-export?

As the name suggests, this is a library to handle importing and exporting data. The django-import-export library supports multiple formats, including xls, csv, json, yaml, and all other formats supported by tablib. It also have a Django admin integration, which is really convenient to use. Pip is the way to go: pip install django-import-export.


2 Answers

It was resolved when I used before_save_instance(self, instance, using_transactions, dry_run)

The function can modify the object before passing it to the model.

class ProjectActivityResource(resources.ModelResource):
    latitude = Field(attribute='latitude', column_name='latitude')
    longitude = Field(attribute='longitude', column_name='longitude')

    class Meta:
        model = ProjectActivity
        fields = ('id', 'project_site', 'name', 'latitude',
                        'longitude', 'date_updated')
        exclude = ('geom')
        export_order = ('id', 'project_site', 'name', 'latitude',
                        'longitude', 'date_updated')

    def before_save_instance(self, instance, using_transactions, dry_run):
        instance.geom = Point(float(instance.longitude), float(instance.latitude))
        return instance
like image 183
Nikko Avatar answered Oct 23 '22 07:10

Nikko


I had some problem, similar to @Nikko. I had a really hard time doing what I wanted, and Nikko pieces of code helped. I'm not completely satisfied with what I've done but it works and maybe it can help some people. It's dirty so if someone wants to explain me what would be the right way, I'm all ears.

This code allows you to import AND export (using django-import-export), from the admin interface, a class containing a PointField (from django-geojson) by storing only the latitude and the longitude in the output file (not the all geojson file).

admin.py :

from leaflet.admin import LeafletGeoAdmin
from import_export import resources
from import_export.fields import Field
from import_export.admin import ImportExportModelAdmin
import json

from django.db import models
from djgeojson.fields import PointField

class SleepSpotResource(resources.ModelResource):
    latitude = Field(attribute='latitude', column_name='latitude')
    longitude = Field(attribute='longitude', column_name='longitude')

    class Meta:
        model = SleepSpot
        fields = ('id','album','title','start_date','end_date','latitude','longitude' )
        exclude = ('geom')
        export_order = ('id','album','title','start_date','end_date','latitude','longitude' )

    def before_save_instance(self, instance, using_transactions, dry_run):
        longitude = float(getattr(instance, 'longitude'))
        latitude = float(getattr(instance, 'latitude'))

        instance.geom = {'type': 'Point', 'coordinates': [longitude, latitude]}
        return instance

    def dehydrate_longitude(self, sleepspot):
        try:
            geomjson = sleepspot.geom
            if type(geomjson) is str:
                geomjson = json.loads(geomjson.replace("\'", "\""))
            return geomjson['coordinates'][0]
        except:
            pass

    def dehydrate_latitude(self, sleepspot):
        try:
            geomjson = sleepspot.geom
            if type(geomjson) is str:
                geomjson = json.loads(geomjson.replace("\'", "\""))
            return geomjson['coordinates'][1]
        except:
            pass

@admin.register(SleepSpot)
class SleepSpotModelAdmin(LeafletGeoAdmin, ImportExportModelAdmin):
    list_display = ('title', 'start_date', 'end_date', 'album')
    resource_class = SleepSpotResource
like image 34
Fritzip Avatar answered Oct 23 '22 05:10

Fritzip