Transformation and management of XML data

[Project description] [Prerequisites] [Examination] [Schedule] [Groups] [Useful links] [Lecturers]

4-week project, November 25 to December 20, 2002.

News

Please use the revised BibTeX files from December 13.
A note has been added on problems determining whether all data in a request to the web service has been received.
Problems have been reported with using the XML libraries with some newer versions of java. Please don't get stuck with such problems, but move on with the essentials of the project. An alternative to using Java is to use stand-alone command-line tools (e.g., this DTD validator) or accessing XSLT through php. Please send us any good solution you might have for this!
Anna will be away December 8-15 and will not be able to answer e-mails.
The exam will take place January 21. See the exam plan below.
Those who want to use a newsgroup for discussions about the project use the newsgroup for the Advanced XML course, named it-c.courses.AXML.

Project description

Background

Lots of data is stored in formats that make it hard to manage the data and to perform the queries one would want. Either because the format is not standardized, or because the specific format lacks software tools. XML formats promise to ease the task of managing, transforming and querying data, due to the large number of tools aimed at supporting XML data. Therefore a current trend is to transform old data formats into an XML format. This also includes the implementation of transformations, queries, etc. for the new XML data.

Core contents

As an example of the above, the project considers transformation and management of bibliographical data (i.e., information on books, journal articles, etc.), which is currently widely stored in the BibTeX format. The core of the project consists of the following tasks:

Extensions

The core project can be extended in many ways, and the groups are free to pursue the extension(s) they find most interesting. Some ideas for extensions are given below.

More advances queries/transformations and schemas for the results. For example, "for each author, list all coauthors and coauthors of coauthors".
Description and implementation of the web service according to the WSDL web service standard, possibly using the SOAP protocol.
Use of other web services to complement the XML-bib data, e.g., find the price of books using Amazon's web service, or search for information on an article using Google's web service.
Make a full interface to the web service, including e.g. uploading of BibTeX files and browsing of results.
Combine several XML-bib files, with overlapping information, to get more complete data than in the original files.
Make a transformation of XML-bib back to BibTeX.
Build a web service interface to an existing web database for BibTeX, e.g. The Collection of Computer Science Bibliographies database.
Write a mini-survey on a selected XML technology, perhaps comparing it with competing technologies (e.g., SOAP, WSDL, XSLT vs XQuery). The survey could focus on both practical and theoretical aspects, and should preferably take the problems in the project as a starting point.

Requirements

To pass the project, the group should solve the core tasks satisfactorily, and document its work in a report.

The report

The report may be in Danish or English. The maximum length of the report is 10 pages plus 5 pages per person in the group. There is also the possibility to add appendices, which will be read at the discretion of the examiners to clarify things in the report. The report must be a self-contained description of the project and the group's solutions, and describe the decisions made and the thoughts behind them. The descriptions, motivations, etc. in the report will be the main parameter in the evaluation of the project. The report must contain:

A clear documentation of the productions. Among other things there must be a description of how to run and test the programs, documentation of the code by comments in the code, names of files and where to find them.
A description of any failures to achieve the goals of the project, including an explanation of the problems encountered.
Citations/inspiration: Many tasks in the project have publicly available solutions. It is allowed to look and learn from these, but all inspiration must be cited in the report, and your own solutions must be made from scratch.

Any extensions made (and their documentation in the report) may increase the grade.

Deadline

The deadline for handing in the project report and programs is December 20, 2002.

Examination

There will be an oral group exam with external censorship in January 2003. The project participants will be individually graded according to the 13-scale.

The exam will take place January 21 and the censor is Torben Bach Pedersen from Aalborg. Where the exam will take place will be announced later.

Time Group

9.10-9.40 Lars

9.40-10.10 Michael

10.10-10.40 Qi

10.50-11.20 Mads Peter

11.20-11.50 Kenneth

12.00-12.30 Bjarke

13.30-14.30 Niels, Mohammed

14.40-15.40 Kasper, Jacob

15.50-17.50 Koffi, Thomas, Janus, Thor

Prerequisites

The students should have experience corresponding to the Introductory Programming and Web Programming courses. In particular, they should know java, and have experience with XML, XSLT, and dynamic web pages.

Groups

Groups should have 3 or 4 members.

Schedule

The teaching will be in the form of 4 lectures of about 1 hour each and 3 individual meetings with each group (up to 45 minutes per group).

Group meeting schedule:

10.00: Kasper, Jacob
11.00: Niels, Mohammed
11.45: Bjarke
13.00: Koffi, Thomas, Janus, Thor
14.00: Lars, Michael, Qi

Time Activity Place

Nov 18, Monday, 12:00 Signing of project agreements 1.45

Nov 25, Monday, 14:00 Introduction 1.03

Nov 27, Wednesday, 10:00 Lecture on parser and schema 1.03

Nov 29, Friday, 10:00 Lecture on queries and XSLT 1.03

Nov 29, Friday, 15:00 Hand in project report fragments (in pdf or ps) for Tuesday meeting e-mail

Dec 2, Monday, 10:00 Lecture on web services and web interface 1.03

Dec. 3, Tuesday, (10-15) 1st individual group meeting 1.03

Dec. 6, Friday, 15:00 Hand in project report fragments (in pdf or ps) for Tuesday meeting e-mail

Dec. 10, Tuesday (10-15) 2nd individual group meeting 1.03

Dec. 13, Friday, 15:00 Hand in project report fragments (in pdf or ps) for Tuesday meeting e-mail

Dec. 17, Tuesday (10-15) 3rd individual group meeting 1.03

Dec 20, Friday, 12:00 Deadline for hand-in Student administration office

Useful links

Web programming course
XML course (Data on the web)
BibTeX format
XML tutorial
Google WSDL interface
Amazon WSDL interface

Teachers

Rasmus Pagh
Office: 1.09
Phone: 38 16 89 34
Email: pagh@it-c.dk

Anna Östlin
Office: 1.09
Phone: 38 16 88 21
Email: annao@it-c.dk

Instructor Rasmus Jørgensen will be helping out with the project. He will answer questions on e-mail (raz@it-c.dk) and on the newsgroup. He will also be available in room 3.15 from 9 to 14 on the following days:

Tuesday Dec. 3rd
Thursday Dec. 5th
Tuesday Dec. 10th (changed!)
Wednesday Dec. 11th
Friday Dec. 13th
Monday Dec. 16th

Time	Group
9.10-9.40	Lars
9.40-10.10	Michael
10.10-10.40	Qi
10.50-11.20	Mads Peter
11.20-11.50	Kenneth
12.00-12.30	Bjarke
13.30-14.30	Niels, Mohammed
14.40-15.40	Kasper, Jacob
15.50-17.50	Koffi, Thomas, Janus, Thor

Time	Activity	Place
Nov 18, Monday, 12:00	Signing of project agreements	1.45
Nov 25, Monday, 14:00	Introduction	1.03
Nov 27, Wednesday, 10:00	Lecture on parser and schema	1.03
Nov 29, Friday, 10:00	Lecture on queries and XSLT	1.03
Nov 29, Friday, 15:00	Hand in project report fragments (in pdf or ps) for Tuesday meeting	e-mail
Dec 2, Monday, 10:00	Lecture on web services and web interface	1.03
Dec. 3, Tuesday, (10-15)	1st individual group meeting	1.03
Dec. 6, Friday, 15:00	Hand in project report fragments (in pdf or ps) for Tuesday meeting	e-mail
Dec. 10, Tuesday (10-15)	2nd individual group meeting	1.03
Dec. 13, Friday, 15:00	Hand in project report fragments (in pdf or ps) for Tuesday meeting	e-mail
Dec. 17, Tuesday (10-15)	3rd individual group meeting	1.03
Dec 20, Friday, 12:00	Deadline for hand-in	Student administration office