Tracking here refers to techniques to ascertain what use and reuse has been made of open educational resources by people after they have been released. The emphasis is on tracking what has happened to a resource: how many times it has been viewed or downloaded, whether it has been copied to another server, whether derivatives have been made. User activity tracking is not in scope here, other than to the extent that it can be assumed that a user will be doing the downloading, copying and remixing (this is touched upon in the chapter on Paradata).
There are several reasons for wanting to track resources, depending on the stakeholder in question.
Of course the assumption behind such reasoning is that resource tracking will actually be able to show evidence of use. However tracking can be something of a double-edged sword, as it may equally show that resources are not being used and that the time and effort spent clearing rights so that they could be edited freely was of little value. It is also clear that tracking should be subordinate to the release of resources, that is, it is necessary to first solve the problems inherent in releasing resources before they can be tracked. In addition, tracking techniques should not limit the ability of users to access and reuse open resources, nor should it interfere with the benefits that resource publishers may wishes to achieve by releasing content as open educational resources.
The initial call for Phase One of the UK OER Programme included the following recommendation1:
"As far as is possible projects will need to track the volume and use of the resources they make available, because: JISC aims to track the outputs of this programme, contributors benefit from access to usage information, and monitoring usage improves resources and services. Projects should put their own tracking and reporting in place, and be prepared to work with JISC to develop these approaches. To support institutions in this, a range of potential measures of success for OER repositories will be published by JISC early in 2009. This data will be essential in determining whether a longer term programme would be useful."
To incentivise the collection of usage data, the Programme offered small grants to projects to measure and report on the use of their content. Few projects tackled tracking in any depth, the one or two notable exceptions were the more mature OER projects based at the University of Oxford, University of Nottingham and the Open University. The reasons for this apparent lack of interest in tracking, and in some cases active resistance, are explored later in this chapter.
Before considering technical approaches to tracking use and reuse it is worth remembering that there are many approaches to tracking that involve little use of technology, for example, asking users to fill in feedback forms, gathering feedback through informal communication and formal interviews. Clearly there are problems associated with these approaches; the effort required may not be justified by the number of responses received, or the sample may be biased by "friends and colleagues" and other eager users who are more likely to respond than more casual users. However, these techniques do make up for what they lack in coverage by increased depth in the data that can be gathered.
If one has access to the log files kept by the web server hosting the OERs then monitoring how often those resources are downloaded from that server is routine since analysis packages have long existed for such tasks on any website.
This information may also be available from hosts such as Flickr, YouTube, SlideShare and other web sites for sharing resources, displayed as a count of the number of times a resource has been viewed on that site, downloaded, or embedded into third-party sites. This information is often also available through the host's API to facilitate automatic monitoring.
Information from the logs includes data about the user's internet domain, client software and the platform being used to access resources. This may be useful if the number of visitors from a specific user group which can be associated with a domain is of interest, for example if the target audience is not in the UK.
The referrer logs also provide information about the link which sent the user to the OERs being downloaded, which is useful in providing information on who is linking to the resources, and on the keyword terms being used to find the resources in search engines.
However the extensive use of caches, web crawling robots (e.g. used by search engines) and the general impossibility of knowing whether a downloaded file was actually viewed, render the results suspect for any use other than qualitative comparison between resources from servers where the logging and analysis routines are known to be similar.
The idea behind using web search as a tracking mechanism is that it is possible to discover where on the open web a resource is available by putting a unique phrase, tag or identifier into the text of the resource and then searching Google or other index for that phrase. The key phrase used should be should be unique to the resource, project or programme depending on the level of aggregation required for the tracking. The Engineering OER pilot project2 proposed a "date-of-birth" code3 for this use made up of three parts: three-letters to identify the project or institution, the date of assignment of the code in format ddmmyy, a unique 5 character string to disambiguate resources assigned codes on the same day. The recommendation was that this should be included in the copyright and licence notice as this text is least likely to be altered. As an alternative to the date of birth code as text, one could use an http URI as an identifier with the advantage that it could act as a link back to the original resource or other useful information, and that people following that link would generate useful tracking information (see refback tracking).
Refback has been suggested by Creative Commons as a means of reuse tracking4. It involves putting a link in the resource being tracked to the site doing the tracking (the two variants are that this may be either the publisher or Creative Commons, i.e. independent and distributed or hosted and centralised). If a curious user follows that link (and the assumption is that occasionally someone will) the tracking site will log the request for the page to which the link goes, included in the log information is the “referrer” i.e. the URL of the page on which the user clicked the link. An application on the tracking site will work through this referrer log and fetch the pages for any URL it does not recognise to ascertain whether they are copies of a resource that it is tracking. It's worth noting that links clicked in on a secure website (https) do not include referrer information if the destination is not also a secure https destination5. To allow tracking of resources located in secure environments the solution is to also make sure the destination is on https.
The idea of using URL redirection7 for tracking is that instead of providing a link that goes directly to a web resource, one provides a link to a service which first logs data for tracking purposes and then redirects the browser to the required resource. Such an approach is common in catalogues and search engines, Google for example uses this technique to track which links from search engine results pages are followed. With the recent rise in popularity of URL shortening services (e.g bit.ly or the twitter t.co shortener), which are in essence just URL redirection services, and growing competition between these services, some have begun to provide tracking functionality which allows their users to monitor the traffic being routed through the shortened URL.
This approach may be applied to OER tracking in two distinct ways: links to the OER may be redirected or links from the OER may be redirected.
It is possible to gather information about how often a resource is accessed by providing a redirected URL as a link to the resource and examining the data gathered through it. This is useful for tracking the use of the resource as a result of specific dissemination or advertising activities (e.g. a listing in some directory or an email announcing the resource). It is less useful for providing general information about how often a resource is accessed, as the direct URL for accessing the resource will soon become known to people who will then be more likely use the direct rather than the indirect URL.
Information about who is using a resource may be gathered by using indirect links in the resource so that anyone following a link from the resource leaves some tracking information. If a suitable link is chosen this can be a reasonable indicator that the resource is actually being used (as opposed to those methods which only show that the resource has been downloaded). The information recorded may include the web location of the OER, which can be valuable if one wants information on whether OERs are being copied and used from sites other than the original host.
A side benefit of this approach is that having a link in the resource to an object on a server under the control of the OER provider may be useful for conveying information after the resource is released. For example, a single pixel transparent gif could be changed to a large red warning indicating that information contained in the resource is no longer valid.
During the course of Phase One of the UK OER Programme it became clear that tracking was not a priority for many of the projects. Even the offer of additional funding for reports on tracking, during Phase Two of the Programme, elicited few examples that went beyond simple reporting of access statistics such as the number of views and downloads from the hosting web server. Of the two meta studies commissioned by the Programme, the Value of Reuse Study8 or the study on Learners Use of Learning Resources9, neither identified significant quantitative data from the projects, though qualitative data was more easily found.
The reason for this may be partially due to a clash in priorities. While funders and other observers had the space to look ahead at how to gather evidence that would be desirable in securing future funding for the sustainable release of OERs, project staff had more immediate concerns. Project teams already had to address many complex issues, e.g. relating to licensing and partnership agreements, before they could start releasing their open educational resources and, clearly, tracking can only take place if content is released in the first place. Also, in the absence of sustained current practice with different approaches to tracking OERS, the long term costs and benefits are unkown. It is arguable that recognition of the importance of tracking is an indicator of mature OER initiatives, that appreciate of the type of evidence that needs to be gathered in order to make a case for sustainable input and investment.
Most of the approaches outlined above involve an additional level of technical complexity which could, in certain circumstances, inhibit both the release and use of OERs. For example, third-party redirection services may fail within the lifetime of the resource, thus breaking the link to the content. Unless one has confidence in the longevity of such services, third-party redirection services should only be used for short-term tracking, e.g. tracking the effectiveness of a time limited dissemination campaign. However the redirect need not be handled by a third party; it could be the resolution service hosted by the OER provider for HTTP-URI identifiers for the resource, that are independent of where the resource itself is hosted. There may also be degradation in the time taken to load the resource in the case of web bugs, as loading them involves an HTTP request and response, which will take time even if the image being used is minimally small, this may be an issue if the server on which they reside is slow to respond.
One of main limitations on how well several of these tracking methods work is the assumption that the resource being tracked will be a web resource: written in HTML, hosted on a web server, viewed and used through a web browser or other internet-connected application. In reality many OERs released through the UK OER Programme were either "old fashioned" formats such as Powerpoint and Word documents, or were multimedia content such as audio or video recordings. While ouch resources are frequently hosted and disseminated from web sites they themselves are not "web native" and their use is more difficult to track.
There are ethical and legal implications involved with several of the tracking approaches outlined above. While the emphasis here is on tracking the use of resources, this necessarily involves tracking the action of individual users, which has privacy implications. The EU e-Privacy Directive10 ("cookie law") may apply to some of the methods discussed in this chapter. Some users may take steps to avoid being tracked, which will render some efforts at resource tracking ineffective, for example URL redirection is sometimes used for nefarious activities and may be flagged as malware by some applications. Perhaps the biggest cause for concern is that tracking methods, that are used for reasons that some users find objectionable, will lead to reputation loss by association. The distinction between tracking the use of a resource and tracking the users of a resource may not be seen as clear cut by everyone involved.
As Helen Beetham has said "openness is the enemy of knowability"11. Tracking would be much easier in the context of a closed centralised system, that requires users to register before accessing content that can not be re-used or copied. However open educational resources stand little chance of thriving in such an environment
The overwhelming findings of the Programme were that appetite for tracking was low and that approaches to resource tracking in the OER domain were immature. However it is encouraging that the more experienced and mature UK OER projects did appear to appreciate the benefits of tracking their content. The Open University's TrackOER project12, for example, is seeking solutions to support its service development and has made rapid progress in investigating web bugs.
The mechanisms for tracking are developing as a component of the digital business models of the big web players, within constraints set by regulatory frameworks such as Data Protection13, the EU cookie directive14 and other privacy laws. It is likely that the accuracy of resource tracking on the web will improve as companies such as Google, Facebook, etc, invest resources in this area, in order to support their business models.
In parallel, it is possible that increased sharing of open usage statistics will encourage people to be less fearful of such data within the context of an open content. More confidence in capturing and interpreting data may also increase the appetite for tracking. Jorum's beta dashboard15, developed in parallel with Phase Three of the UK OER Programme, provides a useful example of open usage statistics, and hopefully reflects the beginnings of a shift in the perceived value of such data.
Related to this final point are emerging developments in aggregating and sharing usage data. At the very basic level this may involve repository managers sharing data information and files, however there are emerging opportunity for recording and sharing paradata.
It is perhaps understandable that projects that are not committed to tracking, are inclined to suggest that it cannot be done effectively. While it is true that no single approach will capture all use and reuse of a resource, and that some use will be hidden from all reasonable attempts at tracking, there is still more to be lost than to be gained from making no attempts to track the use and reuse of open educational resources.
There has been error in communication with booki server. Not sure right now where is the problem.
You should refresh this page.