Using the Web Service¶
This will be a general walk-through of how to use the web service to create, import, and fetch data. First, initialise a new Neo4j database as described in the initialisation instructions. Now, restart the Neo4j server like do:
$NEO4J_HOME/bin/neo4j start
Verify that everything's working by fetching your user profile using the
following Curl command (for these examples I'll be using my own
$USER
, which is mike
):
curl http://localhost:7474/ehri/classes/UserProfile/mike
If all goes well you should see something like:
{
"id" : "mike",
"type" : "UserProfile",
"data" : {
"identifier" : "mike",
"name" : "Mike Bryant"
},
"meta" : {
"gid" : 22
},
"relationships" : {
"lifecycleEvent" : [ {
"relationships" : {
"hasActioner" : [ {
"relationships" : { },
"data" : {
"identifier" : "admin",
"name" : "Admin"
},
"meta" : {
"followers" : 0,
"watching" : 0,
"following" : 0
},
"id" : "admin",
"type" : "Group"
} ]
},
"data" : {
"eventType" : "creation",
"timestamp" : "2015-10-01T15:06:38.411+01:00"
},
"meta" : {
"childCount" : 1
},
"id" : "a2035f24-6845-11e5-92aa-ed4cb7d1dda8",
"type" : "SystemEvent"
} ],
"belongsTo" : [ {
"relationships" : { },
"data" : {
"identifier" : "admin",
"name" : "Administrators"
},
"meta" : {
"childCount" : 1
},
"id" : "admin",
"type" : "Group"
} ]
}
}
This JSON data is a type of graph. It says that:
- The user profile has global ID
mike
, which is the same as the local (non-hierarchical) identifiermike
- The profile is of type
UserProfile
- It has a
belongsTo
relationship to aGroup
item with IDadmin
- There was a
SystemEvent
of typecreation
, "actioned" (or initiated) byadmin
.
Assuming that worked, lets go on and create the following:
- a
Country
item (via the web service RESTful API) - a
Repository
item via EAG XML import .de/) XML import) - a
DocumentaryUnit
item via EAD XML import
Items 1, 2, and 3 here have a hierarchical relationship, which means that we need to create them in order, since the documentary unit item will "belong" to the repository, and the repository to the country item.
Web service usage conventions¶
Before we start, lets just explain a couple of conventions for using the
web service. You'll notice below that any requests that change data,
e.g. POST
, PUT
, DELETE
methods, require an X-User: $USER
header. This identifier the "actioner" to the system and allows the
permission system to determine if they have adequate permissions
(obviously, it's the responsibility of the client application to verify
people are who they say they are, but the web service does not handle
authentication, only authorisation.)
We also use the X-LogMessage
header when we want to tell the system
what we're doing. This will end up as the logMessage
property on the
SystemEvent
item associated with the change.
Creating the country¶
We're going to create the country via the "RESTful" web service API. The
only data we absolutely need for the country item is an ISO-3166-1
alpha-2 (2 letter) code. For now we'll use nl
, for the Netherlands.
We need to send JSON data to the web service and it needs to be in a particular format, namely an object with the following data members:
- a
type
property for the "content type" of the new item - a
data
object property, containing at least the item's mandatory properties - an optional
relationships
object attribute, for nested data members
For a country, we just need type
, and data
, and the only
attribute that needs to be present in data
is the identifier
property containing our two-letter code. So the JSON data we'll send
looks like this:
{
"type": "Country",
"data": {
"identifier": "nl"
}
}
Let's send that via CURL to the /ehri/classes/Country
endpoint,
using application/json
as the content type:
curl -X POST \
-H "X-User: mike" \
-H "Content-type: application/json" \
--data '{"type": "Country", "data": {"identifier": "nl"}}' \
http://localhost:7474/ehri/Country
We should recieve the newly-create item as the response, with more system-created metadata:
{
"id" : "nl",
"data" : {
"identifier" : "nl"
},
"type" : "Country",
"relationships" : {
"lifecycleEvent" : [ {
"id" : "1faabf00-6903-11e5-a63f-d32c8c516fe2",
"data" : {
"timestamp" : "2015-10-02T13:43:04.105+01:00",
"eventType" : "creation"
},
"type" : "SystemEvent",
"relationships" : {
"hasActioner" : [ {
"id" : "mike",
"data" : {
"name" : "mike",
"identifier" : "mike"
},
"type" : "UserProfile",
"relationships" : { },
"meta" : {
"following" : 0,
"followers" : 0,
"watching" : 0
}
} ]
},
"meta" : {
"childCount" : 1
}
} ]
},
"meta" : {
"childCount" : 0,
"gid" : 26
}
}
A few things to note:
- the system-defined global ID is the same as the local identifier property we provided, since countries are top-level items
- there's a
creation
event, actioned by us
Since the global ID is nl
, we can fetch the item from the web
service specifically using:
curl http://localhost:7474/ehri/classes/Country/nl
This should give the same result as when we created it.
Importing a repository via EAG¶
Now we're going to create a repository inside our new country. Since repositories require somewhat more metadata to be useful, we're going to import it via an EAG file. Download and extract the sample EAC, EAD, and EAG XML documents from the samples.tgz archive.
To import eag.xml
we're going to use the /ehri/import/eag
web
service method like so:
curl -X POST \
-H "X-User: mike" \
-H "X-LogMessage: Testing EAG import" \
-H "Content-type: text/xml" \
--data @eag.xml \
http://localhost:7474/ehri/import/eag?scope=nl
We should receive a response like so:
{"message":"Testing EAG import","updated":0,"created":1,"unchanged":0}
This tells us that we created one new item, as expected. The import endpoint is idempotent so we can run the same thing again and it'll tell us that there were no changes.
{"message":"Testing EAG import","updated":0,"created":0,"unchanged":1}
Unfortunately, the import methods don't tell us much about the actual items we've just created, so lets list all the repositories and see what we see there:
curl http://localhost:7474/ehri/classes/Repository
This gives us the somewhat more extensive data:
[
{
"data": {
"identifier": "test-repository",
"typeOfEntity": "organisation"
},
"id": "nl-test_repository",
"meta": {
"childCount": 0,
"gid": 30,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"accessibility": "Use public transport",
"creationProcess": "IMPORT",
"history": "Example Repository Description",
"holdings": "Large",
"identifier": "test-repository#desc",
"languageCode": "eng",
"name": "Test Repository",
"openingTimes": "9-5 All week",
"otherFormsOfName": "Test Repository - Alt Name",
"rulesAndConventions": "ISDIAH",
"typeOfEntity": "organisation"
},
"id": "nl-test_repository.eng-test_repository_desc",
"meta": {
"gid": 31
},
"relationships": {
"hasAddress": [
{
"data": {
"countryCode": "NL",
"email": "info@example.com",
"municipality": "A Town",
"postalCode": "ABC 123",
"street": "Any Street",
"telephone": "12345 678910",
"webpage": "www.example.com"
},
"id": "3320f8ec-6905-11e5-accc-dd70c2facf88",
"meta": {
"gid": 33
},
"relationships": {},
"type": "Address"
}
],
"hasUnknownProperty": [
{
"data": {
"eag_archguide_desc_email_": "Email",
"eag_archguide_desc_webpage_": "Website",
"eag_archguide_identity_repositorid_": "test"
},
"id": "3320f8ee-6905-11e5-accc-dd70c2facf88",
"meta": {
"gid": 32
},
"relationships": {},
"type": "UnknownProperty"
}
]
},
"type": "RepositoryDescription"
}
],
"hasCountry": [
{
"data": {
"identifier": "nl"
},
"id": "nl",
"meta": {
"childCount": 1
},
"relationships": {},
"type": "Country"
}
],
"lifecycleEvent": [
{
"data": {
"eventType": "ingest",
"timestamp": "2015-10-02T13:57:55.686+01:00"
},
"id": "334e98a2-6905-11e5-accc-dd70c2facf88",
"meta": {
"childCount": 1
},
"relationships": {
"hasActioner": [
{
"data": {
"identifier": "mike",
"name": "mike"
},
"id": "mike",
"meta": {
"followers": 0,
"following": 0,
"watching": 0
},
"relationships": {},
"type": "UserProfile"
}
]
},
"type": "SystemEvent"
}
]
},
"type": "Repository"
}
]
We have a JSON list with one item (as expected, since we've only created one repository). A few things to note:
- this time the
lifecycleEvent
isingest
, again withhasActioner
"mike". - there is a
hasCountry
relationship pointing to ournl
country - there is a
describes
relationship to aRepositoryDescription
item with (default)languageCode
valueeng
, that contains the bulk of the data in our EAG file. - the system ID is
nl-test_repository
, and the identifier value istest_repository
.
We can fetch the item directly using its ID via:
curl http://localhost:7474/ehri/classes/Repository/nl-test_repository
Importing a documentary unit via EAD¶
Importing EAD is much the same process, only this time we use the
/ehri/import/ead
method and the repository (with ID
nl-test_repository
) is the scope
item that we are importing
into:
curl -X POST \
-H "X-User: mike" \
-H "X-LogMessage: Testing EAD import" \
-H "Content-type: text/xml" \
--data @ead.xml \
http://localhost:7474/ehri/import/ead?scope=nl-test_repository
Run this and again we get an import log like so:
{"message":"Testing EAD import","updated":0,"created":1,"unchanged":0}
Running GET
on the /ehri/classes/DocumentaryUnit
method lists
the units in the system, giving us this rather verbose output:
[
{
"data": {
"identifier": "test-doc"
},
"id": "nl-test_repository-test_doc",
"meta": {
"childCount": 0,
"gid": 37,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"creationProcess": "IMPORT",
"extentAndMedium": "167 files",
"languageCode": "eng",
"languageOfMaterial": [
"eng",
"fra",
"deu",
"heb",
"ron",
"yid"
],
"levelOfDescription": "collection",
"name": "Test EAD Item",
"scopeAndContent": "This is some test scope and content.",
"sourceFileId": "C00001#ENG"
},
"id": "nl-test_repository-test_doc.eng",
"meta": {
"gid": 38
},
"relationships": {
"hasDate": [
{
"data": {
"description": "1924-1-1 - 1947-12-31",
"endDate": "1947-12-31",
"startDate": "1924-01-01"
},
"id": "de1f5c77-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 42
},
"relationships": {},
"type": "DatePeriod"
},
{
"data": {
"description": "1943-1-1",
"endDate": "1943-01-31",
"startDate": "1943-01-01"
},
"id": "de1f5c79-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 43
},
"relationships": {},
"type": "DatePeriod"
}
],
"hasUnknownProperty": [
{
"data": {
"ead_archdesc_did_langmaterial_language_": [
"English",
"French",
"German",
"Hebrew",
"Romanian",
"Yiddish"
],
"ead_archdesc_did_repository_address_addressline_": [
"Test Address 1",
"Test Address 2"
],
"ead_archdesc_did_repository_corpname_": "Test Corportate Body"
},
"id": "de1f5c75-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 41
},
"relationships": {},
"type": "UnknownProperty"
}
],
"relatesTo": [
{
"data": {
"name": "Test Name",
"type": "creatorAccess"
},
"id": "de1f5c73-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 40
},
"relationships": {},
"type": "AccessPoint"
},
{
"data": {
"name": "Test Corporate Body",
"type": "subjectAccess"
},
"id": "de1f5c71-6908-11e5-8ffa-0da85690eef2",
"meta": {
"gid": 39
},
"relationships": {},
"type": "AccessPoint"
}
]
},
"type": "DocumentaryUnitDescription"
}
],
"heldBy": [
{
"data": {
"identifier": "test-repository"
},
"id": "nl-test_repository",
"meta": {
"childCount": 1,
"watchedBy": 0
},
"relationships": {
"describes": [
{
"data": {
"languageCode": "eng",
"name": "Test Repository"
},
"id": "nl-test_repository.eng-test_repository_desc",
"relationships": {},
"type": "RepositoryDescription"
}
],
"hasCountry": [
{
"data": {
"identifier": "nl"
},
"id": "nl",
"meta": {
"childCount": 1
},
"relationships": {},
"type": "Country"
}
]
},
"type": "Repository"
}
],
"lifecycleEvent": [
{
"data": {
"eventType": "ingest",
"timestamp": "2015-10-02T14:24:11.166+01:00"
},
"id": "de2db45d-6908-11e5-8ffa-0da85690eef2",
"meta": {
"childCount": 1
},
"relationships": {
"hasActioner": [
{
"data": {
"identifier": "mike",
"name": "mike"
},
"id": "mike",
"meta": {
"followers": 0,
"following": 0,
"watching": 0
},
"relationships": {},
"type": "UserProfile"
}
]
},
"type": "SystemEvent"
}
]
},
"type": "DocumentaryUnit"
}
]
Again, stuff to note:
- the generated ID is
nl-test_repository-test_doc
, while the identifier property istest-doc
- the repository is present in the
heldBy
relationship
Additional import options¶
In these examples we POSTed a single XML file (with content-type
text/xml
) to the import methods. This is convenient for single
files, but often you want to import multiple XML files at once into the
same country, repository, or documentary unit (as child items.) In this
case it is possible to create zip or tar archives containing those files
and POST the data as content type application/octet-stream
.
Another (less good) alternative when the web service server is local is
to POST a file containing a list (one entry per line) of local file
paths with content-type text/plain