Category: SWORD v2

Categories
News SWORD v2

Discussing the scope of SWORDv2

As part of the SWORD v2 developments, the Technical Advisory Panel have been busy discussing many aspects of the proposed new version of the standard.  This has been a lively and engaging process.  If you would like to read these discussions and contribute any feedback, you would be very welcome!

One particularly interesting thread came from the project’s technical lead.  The message concerns the scope of SWORD v2 (what areas it should contribute to, ans which it should not):

Hi Folks,

There’s been some great discussion on the list this past week or two, and I thought it might be time for a summary of what looks to me to be a key sticking point: the scope of sword.

There are two distinct sides to this argument as it’s been articulated on this list:

a) That we should adopt the approach of content management API like CMIS or more likely GData

b) That SWORD should be not say anything about what happens to the content once it is sent to the server.

In general, I am against (a) for a number of reasons.  First, I am concerned that the idioms that are associated with GData are not /necessarily/ appropriate.  The hierarchical file system is a common idiom but an idiom nonetheless, and it wouldn’t be SWORD’s place to therefore build itself over the top of it.  CMIS I have a harder time refuting or accepting, so am open to persuasion either way.  Secondly, I don’t see a reason to re-create a content management standard, since they already exist.  SWORD should, instead, provide support for the things that these standards don’t provide for our sector/use cases, while not preventing the use of them.

From a purists perspective of (b) the main thing that SWORD offers, then, is support for Packaging (with a capital P).  This is a valuable addition to the community since it is both common in our sector and expressly not covered at least by GData and I believe not by CMIS (though again, open to correction).  The support for packaging, though, needs to extend to a full CRUD implementation of AtomPub, which is a large part of what the profile attempts to do.  I think we have had some good technical discussion which which will allow the next draft of the profile to do better at that.

In the mean time, there are some grey area parts of the profile, particularly In Progress and Suppress Metadata which are more content management than they are deposit.  I, personally, think these are important; they are light touch, the profile doesn’t mandate the server to obey them, and they help fulfill known use cases.  Likewise the Statement could be viewed as more content management than not, although we have tried to pitch that as more an informational resource rather than an operational one (i.e. read but not write).

What I’m going to suggest for the next draft is as follows:  we’ll put some more time into analysing the appropriate ways of updating and overwriting deposit packages using the feedback on this list.  And we will extend the profile to cover how you would use the SWORD headers to be used in content management operations /if that’s what your implementation wants/ (e.g. how you might use Suppress Metadata or In Progress with GData).  There will, obviously, be plenty of time for comment.

In conclusion: we must constrain the scope of sword to something which doesn’t tread on anyone’s toes and is of value to the community.  Too far one way or the other and we’ll either be superseded or of no value.

Cheers,

Richard

Categories
SWORD v2

Video introduction to SWORD v2

The team over at CottageLabs.com have created a nice 2 1/2 minute technical introductory video to the SWORD v2 standard.  You can view it via their blog:

The narrator is Richard Jones, the Technical Lead for the SWORD v2 initiative.  The video provides a great high-level introduction to the main technical concepts of the standard, and how these fit into the deposit lifecycle.

Categories
SWORD v2

Decisions regarding the challenges of SWORDv2

Following some great recent discussions by the SWORD Technical Advisory Panel, we’re pleased to announce a few decisions that have been made regarding some of the details for the new version 2 of SWORD.  The full email announcing the decisions is shown below, or can be seen in the list archives of the technical advisory group: http://www.mail-archive.com/sword-app-techadvisorypanel@lists.sourceforge.net/msg00105.html

The decisions came about from discussions within the group over the past few weeks.  They relate to the following questions:

  1. Whether the Statement should be embedded in the Deposit Receipt or be a separate document referenced in an atom:link element: In order to allow SWORD v2 to move from a fire-and-forget methodology to one where a SWORD client can interact with the deposit through what we’re calling the ‘deposit lifecycle’, some form of feedback is required where the client can ask the server for details of what has happened to the deposited item(s).  The proposal is to support this via the provision of a ‘statement’.  Think of it a bit like a bank account statement: You can see what has gone into the bank account (deposits), what might have have happened to the deposit (e.g. interest being added), and full details of of the item.The question here, was whether a copy of the statemnet should be given to the SWORD client when it makes the deposit(s), or if the client should ask for a copy of the statement whenever it wants it.
  2. Whether to use OAI-ORE for the Statement format or an Atom Feed (as per CMIS and GData): There is a decision to be made as to how the statement should be formatted.  Should it be formatted as an OAI-ORE resource map, or using an Atom Feed.  There are pros and cons for each method.
  3. How the client and server should negotiate over the format of the content returned by the edit-media link (EM-URI): If multiple formats of statement are allowed, how should the client and server come to an agreement as to which is the best format to send, based upon a combination of the servers capabilities and the clients preferences.  This problem is known as content negotiation.

The full email below outlines these problems, and the decisions made.  The next job is to now attempt the implementation of the standard, and based on the experiences of the developers and initial users, the standard will likely become refined further.

Dear All,

Thanks for your extensive feedback on the various issues that we have been discussing on this list, it has been really valuable for the project team to get this input.  We have, we think, identified 3 particular issues of contention:

1/ Whether the Statement should be embedded in the Deposit Receipt or be a separate document referenced in an atom:link element

2/ Whether to use OAI-ORE for the Statement format or an Atom Feed (as per CMIS and GData)

3/ How the client and server should negotiate over the format of the content returned by the edit-media link (EM-URI)

The project team has gone through each of these issues carefully, and attempted to extract the simplest solutions but with a view to keeping the SWORD 2.0 specification quite open at this stage, so that community best practices can actually inform the standard itself in the long run.

Therefore, we’re proposing the following approaches to these issues:

1/ Whether the Statement should be embedded in the Deposit Receipt or be a separate document referenced in an atom:link element

If the Statement is to be embedded in the Deposit Receipt, then it needs really to be in OAI-ORE form, for the purposes of being clear foreign markup.  Nonetheless, bearing in mind that there is a question as to whether the Statement should be an Atom Feed, it is clear that this solution will not be adequate by itself.  We therefore propose that the standard provided to the project’s funded developers to code against says that an OAI-ORE serialisation MAY be embedded in the Deposit Receipt (the Deposit Receipt will not be required to meet the OAI-ORE spec for being a resource map itself).

Alongside – or instead – of this, there MAY be one or more atom:link elements in the Deposit Receipt which link to an external Statement. These atom:link elements can specify their type attribute to say whether they are an application/rdf+xml or  application/atom+xml;type=feed.  It will be a requirement of the spec that there MUST be an embedded Statement or at least one separate Statement.

Therefore, you may see a Deposit Receipt like:

<atom:entry>
  <atom:link rel="http://purl.org/net/sword/terms/statement" type="application/rdf+xml" href="http://....."/>
  <rdf:RDF>
    <!-- ORE statement goes here -->
  </rdf:RDF>
</atom:entry>

2/ Whether to use OAI-ORE for the Statement format or an Atom Feed (as per CMIS and GData)

Another good reason for the approach in (1) is that this means we can provide different Statement URIs with different type attributes.  We plan to ask developers to produce an ORE and an Atom Feed Statement format under the project funding.  So you may see a Deposit Receipt like:

<atom:entry>
  <atom:link rel="http://purl.org/net/sword/terms/statement" type="application/rdf+xml" href="http://....."/>
  <atom:link rel="http://purl.org/net/sword/terms/statement" type="application/atom+xml;type=feed"href="http://....."/>
   <rdf:RDF>
      <!-- ORE statement goes here -->
   </rdf:RDF>
</atom:entry>

The combination of approaches in (1) and (2) may seem woolly or indecisive, but we believe that we can’t determine in advance which of these approaches is better, and that it should be up to the community of users and implementers to decide which approach works best based on actual usage of the developed software.  Therefore, while the burden of implementation is placed on the funded portion of the project, we expect community driven implementations/usages to favour one approach over another (possibly taking into account things like compatibility with GData and CMIS, or preferring the more semantic web approach of ORE). We can then use this information later in deriving a SWORD spec which is based on best practices.

3/ How the client and server should negotiate over the format of the content returned by the edit-media link (EM-URI)

The Content Negotiation issue arises from the fact that AtomPub requires at most one edit-media URI with a given type to be available in the Atom Entry (Deposit Receipt).  Since the SWORD server may contain multiple files rather than the one file that AtomPub assumes, what this EM-URI returns under GET is unclear.  We initially considered 2 approaches:

a/    A separate HTTP header like Accept-Packaging to allow content negotiation on a package format
b/    A separate HTTP header like Accept-Media-Features to allow general content negotaiton on feature sets

As we discussed, both of these have pros and cons, and none of the approaches to doing this are marked by any best practices, which makes the project team unwilling to commit to anything too complex or substantial, at a risk to the simplicity and overall success of SWORD. Instead we are suggesting adopting a much simpler approach:

The Deposit Receipt can contain already contain a sword:package element (as per SWORD 1.3), and SWORD 2 plans to allow an arbitrary number of such elements.  These elements will describe the packaging formats supported by the server, so the client will know in advance what the capabilities of the server are.  Therefore, instead of engaging in a content negotiation process, the client will just specify a separate HTTP header indicating what package format should be returned.  Whether this header re-uses the Packaging header used during deposit or specifies a new header has yet to be decided.

Hopefully these approaches make sense to the group.  We are interested in how you think these will go down both during the project and beyond in the community, and if there are any obvious problems with what we’re proposing here as the way forward for SWORD.

All the best,

Richard
(On-Behalf-Of the SWORD project team)

Categories
News Repositories SWORD v1 SWORD v2

SWORD wikipedia entry

SWORD now has a wikipedia entry!

http://en.wikipedia.org/wiki/SWORD_(Protocol)

The page does not have much detail on at the moment, so if you have a minute or two, please take a look at the page, and see if you could add / edit / correct / improve / enhance the page.  If you know of any other entries that should link to or reference the SWORD entry, please could you add those in too.

If you have any SWORD-related content or links that you would like to be added to this site (links to implementations, code, documentation, blog entries, papers) please pass them on to info@swordapp.org and we’ll get them added.

Categories
SWORD v2

SWORD Technical Advisory Panel

As part of the development of SWORD v2 a Technical Advisory Panel has been formed.  This panel consists of experts from across the SWORD and general Digital Repository domains, along with experts in related fields. The purpose of the panel is to ensure that the standard develops in a way that meets the needs of its user community, that it exhibits best-practice in the area of Internet standards, that developers are able to work with it, and that it tries to be generic enough to allow interoperability with other types of systems whilst maintaining its focus on repository resource deposit.  The panel consists of people from universities, national libraries, research funders, commercial companies, developers, repository domain experts, and repository managers.

The following people have generously donated their time and expertise to be on this panel:

  • Julie Allinson (The University of York)
  • Tim Brody (University of Southampton)
  • Pablo de Castro (SONEX / Universidad Carlos III de Madrid)
  • Charles Duncan (Intrallect)
  • Reinhard Engels (Harvard University Library)
  • David Flanders (JISC)
  • John Fearns (Symplectic)
  • Kathi Fletcher (Shuttleworth Foundation Fellow)
  • Steve Hitchcock (University of Southampton)
  • Jason Hoyt (Mendeley)
  • Bill Ingram (University of Illinois at Urbana-Champaign)
  • Richard Jones (SWORD Technical Lead)
  • Graham Klyne (University of Oxford)
  • Stuart Lewis (SWORD Community Manager / The University of Auckland Library)
  • Mark MacGillivray (Developer)
  • Andrea Marchitelli (CILEA)
  • Alistair Miles (The Wellcome Trust Centre for Human Genetics)
  • Ben O’Steen (Developer)
  • Glen Robson (National Library of Wales)
  • Richard Rodgers (MIT)
  • Robert Sanderson (LANL)
  • Peter Sefton (Australian Digital Futures Institute, University of Southern Queensland)
  • Nick Sheppard (UKCoRR / Leeds Metropolitan)
  • Eddie Shin (MediaShelf)
  • Alec Smecher (Public Knowledge Project)
  • Adrian Stevenson (UKOLN)
  • Ian Stuart (Repository Junction / EDINA)
  • Ed Summers (Library of Congress)
  • David Tarrant (University of Southampton)
  • Robin Taylor (The University of Edinburgh)
  • Graham Triggs (BioMed Central)
  • Alex Wade (Microsoft External Research)
  • Paul Walk (UKOLN)
  • Simeon Warner (arXiv)
  • Scott Wilson (CETIS)
  • Nathan Yergler (Creative Commons)

In the interests of openness, the group discussions are being archived in an open mail archive: http://www.mail-archive.com/sword-app-techadvisorypanel@lists.sourceforge.net/

Categories
SWORD v2

SWORDv2 Project Plan – Timeline

This is the third in a series of blog posts outlining the SWORDv2 project plan.  This blog post details the proposed timeline of when each work package will take place.

The project is split into two phases, an initial research and specification phase, followed by a development and test phase.  During this time a second stream of advocacy and general community management will be undertaken.  There will also be a stream of work for project support.

Phase One: November 2010 – January 2011

  • Technical
    • Work package 1: Compile use cases
    • Work package 2: Analysis of white paper
    • Work package 4: Creation of prototype SWORD v2 specification
  • Community
    • Work package 3: Initial community formation
  • Project support
    • Work package 10: Project dissemination
    • Work package 11: Project administration

Phase Two: February 2011 – April 2011

  • Technical
    • Work package 5: Server implementations
    • Work package 6: Client implementations
  • Community
    • Work package 7: Instructional documentation and ongoing community management
    • Work package 9: Support the JISCDepo projects
  • Project support
    • Work package 8: Develop a sustainability plan
    • Work package 10: Project dissemination
    • Work package 11: Project administration
Categories
SWORD v2

SWORDv2 Project Plan – Workplans

This is the second in a series of blog posts that is outlining the project plan for the SWORDv2 project.  This post will describe the work packages that will be undertaken over the next 6 months to complete the project.

Work package 1: Compile use cases

  • The project will work with the SONEX group to gather, document, and publicise relevant use cases that fit with the overarching principles of SWORD v2. These use cases will be used to ensure that the SWORD v2 standard meets the requirements of the repository community.  In addition the community manager will ensure that the repository manager community is made aware of the project, and is encouraged to participate in the collection use cases.

Work package 2: Analysis of white paper

  • As a concluding part to the SWORD 3 project, the Technical Lead wrote a white paper that outlined the potential requirements of the SWORD v2 standard. This white paper was given wide publicity at the Open Repositories 2010 conference and was placed online on the JISCPress site (http://sword2depositlifecycle.jiscpress.org/) for comment by the community.The feedback received from the community needs to be analysed, and once combined with the use cases from the SONEX group it will be written-up to create a document that will define the use cases, requirements, and outputs of the SWORD v2 project. The document will be used to describe to the community what the standard will do and why it will do it, and will give rise to the standard itself in work package 4.

Work package 3: Initial community formation

  • While the analysis of the white paper is being undertaken the Community Manager will need to perform work to start to form an initial community around the SWORD v2 work. This first phase of community building will be centered around publicity of the project including a project web site and blog (to supplement the current swordapp.org site), an elevator pitch to explain the aims of the project to the community, and the creation of an effective communications channel to allow dissemination, discussion and feedback to easily take place.A technical advisory panel will be created that consists of project staff members, and developers and repository managers from the worldwide repository community. They will come from a cross section the different repository platforms and user types. Whilst all aspects of the project will be open for comment by any interested party, the technical advisory panel will be consulted closely at all times to ensure the project is meeting the requirements of the repository community.

Work package 4: Creation of prototype SWORD v2 specification

  • Following the analysis of the white paper, the creation of the report and the formation of the initial community, a prototype SWORD v2 specification will be written. The specification will detail the SWORD v2 protocol to a level where it can be discussed, evaluated, and implemented.The specification will be considered a draft as it is envisaged that during the later implementation phase that the specification will change in light of user experience and evaluation.

Work package 5: Server implementations

  • Once the prototype specification has been written, a server implementation will be required. This will be used in conjunction with the client implementations (work package 6) to test the specification and evaluation it against the use cases and requirements as specified earlier in the project.  The server implementations should attempt to provide some mechanism by which client requests can be validated.  All resultant code will be released with a suitable open source licence.Three server implementations will be created, one each for DSpace, EPrints, and Fedora.  If appropriate and possible, other server implementations will be encouraged with technical support from the project.

Work package 6: Client implementations

  • Once the prototype specification has been written, client implementation will be required. This will be used in conjunction with the server implementations (work package 5) to test the specification and evaluation it against the use cases and requirements as specified earlier in the project.  The client implementations should attempt to provide some mechanism by which they can be used to validate a sword endpoint.There will be 4 client implementations created, to offer a highly heterogeneous environment within which to test the use cases and requirements.  They are also designed to offer a series of multi-language software libraries for easy use within other systems not dealt with here. All resultant code will be released with a suitable open source licence.  The clients will include Java, PHP, Ruby and Phython code.

Work package 7: Instructional documentation and ongoing community management

  • In order to be empowered to interact with the SWORD v2 standard the community will require instructional documentation to make use of the proposed new standard. This will take the form of code examples, training materials, and support.The community will need facilitation to ensure it interacts with the standard and the demonstration implementations. This will be achieved through the provision of support, advocacy and publicity of the standard the implementations.As well as ensuring the user and developer communities adopt SWORD v2 and feel ownership of it, the work of advocating, educating and promoting SWORD v1. This will be achieved by maintaining the SWORD v1 website, and by seeking further opportunities to teach ‘The SWORD Course’ at suitable events.

Work package 8: Develop a sustainability plan

  • A viable sustainability plan is required to ensure the ongoing development and maintenance of the SWORD protocol standard,the implementations, and the advocacy.

Work package 9: Support the JISCDepo projects

  • The JISCDepo programme (#jiscdepo) is a suite of projects that work in the area of repository deposit. This work package will support the JISCDepo projects to use SWORD v2 where applicable, and ensure the SWORD v2 standard meets any suitable requirements that they have.The work will be undertaken by UKOLN through the DevCSI infrastructure. In addition to supporting the JISCDepo programme, they will support the use of SWORD in the wider repository programme of projects funded by JISC. If appropriate, the involvement of DevCSI will be used to run a community development event to bring together developers to experiment and develop with the SWORD v2 standard.

Work package 10: Project dissemination

  • In addition to the advocacy and community development of SWORD v2 through the use of the project website, blog, and other activities, the project will also disseminate its findings, the developments it has created, and encourage community interaction by attending major repository events.  Additionally the project will seek to work with existing support networks such as the RSP to deliver training and and advocacy through their programmes of events.

Work package 11: Project administration

  • The project will require standard JISC project management activities to take place. These include formal project plans, reports, budgets, and attendance at relevant programme meetings. Overheads and administration costs will be required for UKOLN to support these activities and the general running of the project.
Categories
SWORD v2

SWORDv2 Project Plan – staff introductions

The SWORDv2 project is now underway.  The next few blog posts will outline the project plan that is being followed.  The first of these blog posts introduces the core team members.  There are three people leading the project, each with a different area of responsibility.  If you ever have any questions about the project, please feel free to contact these people directly:

Technical Lead: Richard Jones
The technical lead is responsible for leading the development of the SWORDv2 standard.  This involves analysing the current standard and its shortfalls, looking at how the standard should evolve, and surveying the technical landscape to ensure that the proposed solution fits with other systems.

Richard has been involved with the technical specification of all the past SWORD versions, and was joint creator of the current DSpace implementation of the SWORD standard.  He has worked extensively with the SWORD and AtomPub standards, and in the area of repository interoperability, particularly between research information systems such as Symplectic Elements and Frida (the Norwegian national research system).  Richard is also a member of the Scholarly Output Notification and Exchange (SONEX) working group, which works in collecting and analysing use-cases surrounding repository deposit.  He is also chair of the DevCSI Developer Focus group, and a founder member of the DSpace Committer group.

Community Manager: Stuart Lewis
The role of the community manager is to foster a community around the SWORD v2 standard to help it evolve, meet the user requirements, encourage implementations and uptake, perform advocacy and training, and create a cohesive community that will take ownership of the standard allowing it to develop in a sustainable fashion.

Stuart Lewis works for The University of Auckland Library in New Zealand as their Digital Development Manager. He has played a major part of all the past SWORD projects, initially by being the joint creator of the DSpace and Fedora implementations, but also by writing clients such as the Java web client, the PHP API, and the Facebook client. More recently he has contributed by creating the EasyDeposit SWORD client creation tookit, and for writing ‘The SWORD Course’ that was first delivered at the Open Repositories 2010 conference to an audience of 55 international participants.

Stuart blogs regularly about SWORD developments (http://blog.stuartlewis.com/tag/sword/) and has co-authored two papers concerning SWORD:
– Allinson, J., François, S., Lewis, S. SWORD: Simple Web-service Offering Repository Deposit, Ariadne, Issue 54, January 2008. Online at http://www.ariadne.ac.uk/issue54/allinson-et-al/
– Lewis, S., Hayes, L., Newton-Wade, V., Corfield, A., Davis, R., Donohue, T., Wilson, S., (2009) If SWORD is the answer, what is the question?: Use of the Simple Web-service Offering Repository Deposit protocol, Program: electronic library and information systems, Vol. 43 Issue 4, pp.407 – 418. Online at http://hdl.handle.net/2292/5315

Project Director: Paul Walk
The project director is responsible for directing the overall direction of the project, and acting as a contact point between the project funder and the project itself.

Paul Walk has worked predominately in Higher Education in the UK since the early nineties. Starting in the academic library service at the University of North London, he went on to develop Web and intranet systems and take the technical lead in establishing the University’s VLE service in 1997. Paul has been involved in many community and standards activities, notably with JA-SIG, the International DOI Foundation and IMS Enterprise, and was one of the founders of the XCRI specification for course descriptions which has been adopted widely across Europe. Paul joined UKOLN at the University of Bath in 2006 and became Deputy Director in 2010, in which role he has oversight of research and development. He has guided the development of UKOLN as a JISC Innovation Support Centre and the founding of a sustainable community of developers in HE through the JISC-funded DevCSI project.

As well as the three core project staff, there is a small team of technical experts who will perform implementations of the SWORDv2 standard, and a Technical Advisory Panel who will help guide the development of the standard.  More introductions will take place in future blog posts…

Categories
News SWORD v2

SWORDv2 and CRUD

In case you haven’t seen it, there is a great post on the DepositMO blog entitled ‘Repository deposit turns to CRUD‘.  The post provides a good introduction to CRUD, explains how this fits in the with the requirements of the DepositMO project, and how this relates to some of the rationale for the SWORDv2 project.  The SWORDv2 project is lucky to have the close involvement of key DepositMO project staff.

DepositMO is a JISC project creating a repository deposit workflow connecting the user’s computer desktop, especially popular apps such as MS Office, with digital repositories based on EPrints and DSpace. DepositMO involves teams from the University of Southampton and Edinburgh University, and will liaise closely with Microsoft.

Categories
News Repositories SWORD v2

SWORDv2 project receives funding from JISC

We’re pleased to announce some great news…

The SWORDv2 Project has been funded by JISC, under the Information Environment 2011 Programme, to extend repository deposit to cover the wider scholarly communication infrastructure. The project will develop a second generation of the SWORD deposit protocol that will enable it to encompass a wider set of systems within the scholarly communication infrastructure, and to allow active management of artefacts as they change throughout their lifetime.

The original SWORD projects dealt with creating new repository resources by package deposit – a simple case which was at the root of their success but which also represented a key limitation. This method of deposit could be summed up as ‘fire-and-forget’. SWORD supports the deposit of the content, but once it is deposited, the user of a SWORD client is unable to track the progress of the item through any workflows, make alterations or updates to the content, or to delete it.

The next version of SWORD will push the standard towards supporting a full deposit lifecycle for all types of scholarly systems by specifying and implementing update, retrieve and delete extensions to the specification. This will enable these systems to be integrated into a broader range of other systems within the scholarly infrastructure, by supporting an increased range of behaviours and use cases.

The project will deliver a new technical standard for the SWORDv2, repository implementations for DSpace, EPrints, and Fedora, and four client API libraries.

The first generation of the SWORD protocol was developed in the UK with funds from JISC and support from UKOLN, and has been adopted worldwide with acclaim. The project won an award for the most innovative project at the JISC Repositories and Preservation conference in 2009. The standard has gone on to be implemented in all major open source repository platforms, and has clients created in various forms ranging from Facebook to Microsoft Word.

We’ll post further details in the next couple of weeks concerning the launch of the project, further details about the scope of the project, and information about how to get involved.