Req:dataimport
From FOSSWiki
Import & Export Module-Version 1.0
Contents |
Aims & Objectives
Nowadays there are so many governments and NGOs involved in gathering massive amounts of disaster data which are spread all over the world. And it would certainly be remarkable if all of these data can be accessed through one disaster management system. The main objective of this module is to intelligently import and merge external data sources into Sahana. Initially we concentrate on importing disaster data which is in CSV type file format. And as extension it will handle other type of external data sources such as XML, relational, etc…
Scope of the Module
Allows to easily upload information which are related to people (victims), Camps and Organizations from other external data files which must be in ‘comma-separated-values’ format (CSV) into Sahana. But in Next version, it will handle XML, relational and other formats.
Controllable features for data uploading: provide ability to select primary type(whether people,camp or organizations) of the data in the import file.
Ability to manage data import. Here if required user can select which data from the import file should be imported and which should be ignored. When a perticular primary type is selected(say people)the external data file may have many attributes which we may not require. In that case, we have the ability to decide what attribute (fields) we require from the external data file.
Create custom data fields for any additionally needed data fields which are not currently being used by Sahana but which are needed to be stored for future reference/access.
In this first version, we ignore the relationships between entities (people,camp and Organizations) and only focus on instances of these entities.but in the next version module will handle relationships.
Ability to provide intelligent mapping of import data fields with Sahana data fields.
Flexibility to save earlier field mappings to support reusability. Keep track of error handling: Check whether data type of each import fields is in correct data type and format, all required fields of Sahana database have been filled etc… View the summary report of successfulness of data importation.
Deliverables of the Import Module
Version 1: Import data from a CSV file which contains details about persons.
Anticipated Delivery: Completed.
Version 1.0.1: Import data from a CSV file which contains details about Camps.
Anticipated Delivery: Completed.
Version 1.0.2: Import data from a CSV file which contains details about Organizations.
Anticipated Delivery: Completed.
Version 1.1: Provide ability to read XML file and identify the tags. If such file doesn't have any XSD or DTD tags, then read the file anyway and populate that data into Sahana.
Anticipated Delivery: End of the 2 nd week of May
Version 2: Import data from XML sources into Sahana. (Here consider XML +schema)
Anticipated Delivery: End of the 2nd week of July Version 3: Import data from Relational sources into Sahana.
Anticipated Delivery: End of the 4th week of July
User Interfaces
1st Interface – Used to Upload the CSV file. It contains details such as whether 1st row contains field names or not.
2nd Interface – Used to gather more details about CSV data. Such as if it contains details of people, Organizations details or Camp details and location details.
3rd Interface contains details about field mapping with Sahana fields. This module initially guesses and allows user to change the performed mapping.
Note that if external imported file doesn't contain header row, then 'Header Row' content won't be displayed.
Next provide validation of the imported data by comparing data format and type.And Verify that data to be imported are in correct format.
Finally provide the summary report of imported process.
Screen Shots
Home Page :
