Using the Web Service¶
This will be a general walk-through of how to use the web service to create, import, and fetch data. First, initialise a new Neo4j database as described in the initialisation instructions. Now, restart the Neo4j server like do:
$NEO4J_HOME/bin/neo4j start
Verify that everything's working by fetching your user profile using the
following Curl command (for these examples I'll be using my own
$USER, which is mike):
curl http://localhost:7474/ehri/classes/UserProfile/mike
If all goes well you should see something like:
{
"id" : "mike",
"type" : "UserProfile",
"data" : {
"identifier" : "mike",
"name" : "Mike Bryant"
},
"meta" : {
"gid" : 22
},
"relationships" : {
"lifecycleEvent" : [ {
"relationships" : {
"hasActioner" : [ {
"relationships" : { },
"data" : {
"identifier" : "admin",
"name" : "Admin"
},
"meta" : {
"followers" : 0,
"watching" : 0,
"following" : 0
},
"id" : "admin",
"type" : "Group"
} ]
},
"data" : {
"eventType" : "creation",
"timestamp" : "2015-10-01T15:06:38.411+01:00"
},
"meta" : {
"childCount" : 1
},
"id" : "a2035f24-6845-11e5-92aa-ed4cb7d1dda8",
"type" : "SystemEvent"
} ],
"belongsTo" : [ {
"relationships" : { },
"data" : {
"identifier" : "admin",
"name" : "Administrators"
},
"meta" : {
"childCount" : 1
},
"id" : "admin",
"type" : "Group"
} ]
}
}
This JSON data is a type of graph. It says that:
- The user profile has global ID
mike, which is the same as the local (non-hierarchical) identifiermike - The profile is of type
UserProfile - It has a
belongsTorelationship to aGroupitem with IDadmin - There was a
SystemEventof typecreation, "actioned" (or initiated) byadmin.
Assuming that worked, lets go on and create the following:
- a
Countryitem (via the web service RESTful API) - a
Repositoryitem via EAG XML import .de/) XML import) - a
DocumentaryUnititem via EAD XML import
Items 1, 2, and 3 here have a hierarchical relationship, which means that we need to create them in order, since the documentary unit item will "belong" to the repository, and the repository to the country item.
Web service usage conventions¶
Before we start, lets just explain a couple of conventions for using the
web service. You'll notice below that any requests that change data,
e.g. POST, PUT, DELETE methods, require an X-User: $USER
header. This identifier the "actioner" to the system and allows the
permission system to determine if they have adequate permissions
(obviously, it's the responsibility of the client application to verify
people are who they say they are, but the web service does not handle
authentication, only authorisation.)
We also use the X-LogMessage header when we want to tell the system
what we're doing. This will end up as the logMessage property on the
SystemEvent item associated with the change.
Creating the country¶
We're going to create the country via the "RESTful" web service API. The
only data we absolutely need for the country item is an ISO-3166-1
alpha-2 (2 letter) code. For now we'll use nl, for the Netherlands.
We need to send JSON data to the web service and it needs to be in a particular format, namely an object with the following data members:
- a
typeproperty for the "content type" of the new item - a
dataobject property, containing at least the item's mandatory properties - an optional
relationshipsobject attribute, for nested data members
For a country, we just need type, and data, and the only
attribute that needs to be present in data is the identifier
property containing our two-letter code. So the JSON data we'll send
looks like this:
{
"type": "Country",
"data": {
"identifier": "nl"
}
}
Let's send that via CURL to the /ehri/classes/Country endpoint,
using application/json as the content type:
curl -X POST \
-H "X-User: mike" \
-H "Content-type: application/json" \
--data '{"type": "Country", "data": {"identifier": "nl"}}' \
http://localhost:7474/ehri/Country
We should recieve the newly-create item as the response, with more system-created metadata:
{
"id" : "nl",
"data" : {
"identifier" : "nl"
},
"type" : "Country",
"relationships" : {
"lifecycleEvent" : [ {
"id" : "1faabf00-6903-11e5-a63f-d32c8c516fe2",
"data" : {
"timestamp" : "2015-10-02T13:43:04.105+01:00",
"eventType" : "creation"
},
"type" : "SystemEvent",
"relationships" : {
"hasActioner" : [ {
"id" : "mike",
"data" : {
"name" : "mike",
"identifier" : "mike"
},
"type" : "UserProfile",
"relationships" : { },
"meta" : {
"following" : 0,
"followers" : 0,
"watching" : 0
}
} ]
},
"meta" : {
"childCount" : 1
}
} ]
},
"meta" : {
"childCount" : 0,
"gid" : 26
}
}
A few things to note:
- the system-defined global ID is the same as the local identifier property we provided, since countries are top-level items
- there's a
creationevent, actioned by us
Since the global ID is nl, we can fetch the item from the web
service specifically using:
curl http://localhost:7474/ehri/classes/Country/nl
This should give the same result as when we created it.
Importing a repository via EAG¶
Now we're going to create a repository inside our new country. Since repositories require somewhat more metadata to be useful, we're going to import it via an EAG file. Download and extract the sample EAC, EAD, and EAG XML documents from the samples.tgz archive.
To import eag.xml we're going to use the /ehri/import/eag web
service method like so:
curl -X POST \
-H "X-User: mike" \
-H "X-LogMessage: Testing EAG import" \
-H "Content-type: text/xml" \
--data @eag.xml \
http://localhost:7474/ehri/import/eag?scope=nl
We should receive a response like so:
{"message":"Testing EAG import","updated":0,"created":1,"unchanged":0}
This tells us that we created one new item, as expected. The import endpoint is idempotent so we can run the same thing again and it'll tell us that there were no changes.
{"message":"Testing EAG import","updated":0,"created":0,"unchanged":1}
Unfortunately, the import methods don't tell us much about the actual items we've just created, so lets list all the repositories and see what we see there:
curl http://localhost:7474/ehri/classes/Repository
This gives us the somewhat more extensive data:
[
{
"data": {
"identifier": "test-repository",
"typeOfEntity": "organisation"
},
"id": "nl-test_repository",
"meta": {
"childCount": 0,
"gid": 30,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"accessibility": "Use public transport",
"creationProcess": "IMPORT",
"history": "Example Repository Description",
"holdings": "Large",
"identifier": "test-repository#desc",
"languageCode": "eng",
"name": "Test Repository",
"openingTimes": "9-5 All week",
"otherFormsOfName": "Test Repository - Alt Name",
"rulesAndConventions": "ISDIAH",
"typeOfEntity": "organisation"
},
"id": "nl-test_repository.eng-test_repository_desc",
"meta": {
"gid": 31
},
"relationships": {
"hasAddress": [
{
"data": {
"countryCode": "NL",
"email": "info@example.com",
"municipality": "A Town",
"postalCode": "ABC 123",
"street": "Any Street",
"telephone": "12345 678910",
"webpage": "www.example.com"
},
"id": "3320f8ec-6905-11e5-accc-dd70c2facf88",
"meta": {
"gid": 33
},
"relationships": {},
"type": "Address"
}
],
"hasUnknownProperty": [
{
"data": {
"eag_archguide_desc_email_": "Email",
"eag_archguide_desc_webpage_": "Website",
"eag_archguide_identity_repositorid_": "test"
},
"id": "3320f8ee-6905-11e5-accc-dd70c2facf88",
"meta": {
"gid": 32
},
"relationships": {},
"type": "UnknownProperty"
}
]
},
"type": "RepositoryDescription"
}
],
"hasCountry": [
{
"data": {
"identifier": "nl"
},
"id": "nl",
"meta": {
"childCount": 1
},
"relationships": {},
"type": "Country"
}
],
"lifecycleEvent": [
{
"data": {
"eventType": "ingest",
"timestamp": "2015-10-02T13:57:55.686+01:00"
},
"id": "334e98a2-6905-11e5-accc-dd70c2facf88",
"meta": {
"childCount": 1
},
"relationships": {
"hasActioner": [
{
"data": {
"identifier": "mike",
"name": "mike"
},
"id": "mike",
"meta": {
"followers": 0,
"following": 0,
"watching": 0
},
"relationships": {},
"type": "UserProfile"
}
]
},
"type": "SystemEvent"
}
]
},
"type": "Repository"
}
]
We have a JSON list with one item (as expected, since we've only created one repository). A few things to note:
- this time the
lifecycleEventisingest, again withhasActioner"mike". - there is a
hasCountryrelationship pointing to ournlcountry - there is a
describesrelationship to aRepositoryDescriptionitem with (default)languageCodevalueeng, that contains the bulk of the data in our EAG file. - the system ID is
nl-test_repository, and the identifier value istest_repository.
We can fetch the item directly using its ID via:
curl http://localhost:7474/ehri/classes/Repository/nl-test_repository
Importing a documentary unit via EAD¶
Importing EAD is much the same process, only this time we use the
/ehri/import/ead method and the repository (with ID
nl-test_repository) is the scope item that we are importing
into:
curl -X POST \
-H "X-User: mike" \
-H "X-LogMessage: Testing EAD import" \
-H "Content-type: text/xml" \
--data @ead.xml \
http://localhost:7474/ehri/import/ead?scope=nl-test_repository
Run this and again we get an import log like so:
{"message":"Testing EAD import","updated":0,"created":1,"unchanged":0}
Running GET on the /ehri/classes/DocumentaryUnit method lists
the units in the system, giving us this rather verbose output:
[
{
"data": {
"identifier": "test-doc"
},
"id": "nl-test_repository-test_doc",
"meta": {
"childCount": 0,
"gid": 37,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"creationProcess": "IMPORT",
"extentAndMedium": "167 files",
"languageCode": "eng",
"languageOfMaterial": [
"eng",
"fra",
"deu",
"heb",
"ron",
"yid"
],
"levelOfDescription": "collection",
"name": "Test EAD Item",
"scopeAndContent": "This is some test scope and content.",
"sourceFileId": "C00001#ENG"
},
"id": "nl-test_repository-test_doc.eng",
"meta": {
"gid": 38
},
"relationships": {
"hasDate": [
{
"data": {
"description": "1924-1-1 - 1947-12-31",
"endDate": "1947-12-31",
"startDate": "1924-01-01"
},
"id": "de1f5c77-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 42
},
"relationships": {},
"type": "DatePeriod"
},
{
"data": {
"description": "1943-1-1",
"endDate": "1943-01-31",
"startDate": "1943-01-01"
},
"id": "de1f5c79-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 43
},
"relationships": {},
"type": "DatePeriod"
}
],
"hasUnknownProperty": [
{
"data": {
"ead_archdesc_did_langmaterial_language_": [
"English",
"French",
"German",
"Hebrew",
"Romanian",
"Yiddish"
],
"ead_archdesc_did_repository_address_addressline_": [
"Test Address 1",
"Test Address 2"
],
"ead_archdesc_did_repository_corpname_": "Test Corportate Body"
},
"id": "de1f5c75-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 41
},
"relationships": {},
"type": "UnknownProperty"
}
],
"relatesTo": [
{
"data": {
"name": "Test Name",
"type": "creatorAccess"
},
"id": "de1f5c73-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 40
},
"relationships": {},
"type": "AccessPoint"
},
{
"data": {
"name": "Test Corporate Body",
"type": "subjectAccess"
},
"id": "de1f5c71-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 39
},
"relationships": {},
"type": "AccessPoint"
}
]
},
"type": "DocumentaryUnitDescription"
}
],
"heldBy": [
{
"data": {
"identifier": "test-repository"
},
"id": "nl-test_repository",
"meta": {
"childCount": 1,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"languageCode": "eng",
"name": "Test Repository"
},
"id": "nl-test_repository.eng-test_repository_desc",
"relationships": {},
"type": "RepositoryDescription"
}
],
"hasCountry": [
{
"data": {
"identifier": "nl"
},
"id": "nl",
"meta": {
"childCount": 1
},
"relationships": {},
"type": "Country"
}
]
},
"type": "Repository"
}
],
"lifecycleEvent": [
{
"data": {
"eventType": "ingest",
"timestamp": "2015-10-02T14:24:11.166+01:00"
},
"id": "de2db45d-6908-11e5-8ffa-0da85690eef2",
"meta": {
"childCount": 1
},
"relationships": {
"hasActioner": [
{
"data": {
"identifier": "mike",
"name": "mike"
},
"id": "mike",
"meta": {
"followers": 0,
"following": 0,
"watching": 0
},
"relationships": {},
"type": "UserProfile"
}
]
},
"type": "SystemEvent"
}
]
},
"type": "DocumentaryUnit"
}
]
Again, stuff to note:
- the generated ID is
nl-test_repository-test_doc, while the identifier property istest-doc - the repository is present in the
heldByrelationship
Additional import options¶
In these examples we POSTed a single XML file (with content-type
text/xml) to the import methods. This is convenient for single
files, but often you want to import multiple XML files at once into the
same country, repository, or documentary unit (as child items.) In this
case it is possible to create zip or tar archives containing those files
and POST the data as content type application/octet-stream.
Another (less good) alternative when the web service server is local is
to POST a file containing a list (one entry per line) of local file
paths with content-type text/plain