The purpose of this article has been not so much a review of PATOIS as much as an opportunity to reflect on the process by which PATOIS was constructed. In reflecting on this experience, it has been our intention to encourage and inform the development of similar projects in the future. We can only offer a few provisional insights. A 'warts-and-all' evaluation of PATOIS in the long term would be welcome and would certainly be more useful, but is still several years off. In the interim, we can identify those themes that have given us the clearest insight into, and most concerns about, teaching and learning with network-based resources. Issues of accessibility, evaluation and review have been foremost, while the sheer effort involved in creating resources should not be underestimated. The role of computer-based learning within broader curricula - or not - also bears some attention, particularly when it involves finding the correct medium for the subject under discussion. We can also identify a few themes that may be useful to evaluate PATOIS in the future.
There is something truly paradoxical about ensuring that web sites are accessible. On the one hand, it is very easy to ensure accessibility from the outset; on the other hand it can be very difficult to resolve once certain key decisions have been made. In authoring PATOIS, we have largely compiled all the HTML using nothing more complicated than a text editor and some guide books. This approach offers a slow start in comparison with the sorts of web-editing software that is available. PATOIS is in fact rather conservative; only one page in the whole tutorial requires a special plug-in, and this is only because it refers to a virtual reality model in the pages of Internet Archaeology, not because we have created such a model ourselves. Legislation and professionalism require PATOIS to have a standards-based approach. There is a huge variation in the sorts of technologies available to students and universities and, as we have seen the contexts of effective learning with Internet-based resources, these are unpredictable. Conformance with standards is one of the few mechanisms to reduce this variability, and ensure that Internet-based teaching does not conspire to raise new barriers to learning.
"Good digital content doesn't come cheaply, nor should we expect it to. For computer-based learning to be really effective, we need to recognise that it is not a cheap alternative for distributing lecture notes."
Creating truly accessible Internet resources requires planning, and can certainly be more time consuming than may first appear. Indeed, the time and effort involved in creating high-quality learning resources should not be under-estimated. An anecdotal statistic came to our attention at the start of the project, and although we have been unable to substantiate it, it nonetheless bears repeating. According to this statistic, one major distance learning provider in the UK sets aside eight hundred hours of production time for every one hour of contact time. PATOIS is not in the business of creating television programmes or distributing video, and the production time for PATOIS is nothing close to this. Even so, it is worth recognising that good digital content doesn't come cheaply, nor should we expect it to. For computer-based learning to be really effective, we need to recognise that it is not a cheap alternative for distributing lecture notes.
Although computer-based learning can be flexible, imaginative and challenging, it is hard to imagine undergraduate curricula in archaeology ever being completely delivered through the campus network. The PATOIS topics are inherently computer-based, and even here it is clear that learning is enhanced by other forms of engagement. If the best learning experiences from PATOIS include other, conventional interactions, then we can detect a clear signal about good practice in the use of all computer-based learning: good practice depends on fitness for purpose. There will be subjects which either cannot be taught by computer or can only be taught poorly. Our scarce resources of time and money should be invested in mechanisms that will give the best result for our students, not the ones that use the latest technology for its own sake. There is little to be gained from expensive IT solutions to problems already resolved by more conventional, less expensive, means.
The PATOIS experience has shown very clearly the difference between putting content online, and developing teaching materials. The approach used by the PATOIS team tends to be pedagogically conservative: there are aims and objectives, introductions and revision sections. There is a structured engagement with datasets that requires work on the part of the student, laying out a course of study that is to be followed for several hours at a time. This differs from some educational resources where the students are encouraged to explore freely with digital resources, phrasing their own questions and interests in a dialogue with the published content. In this respect, PATOIS draws heavily from the experience of TLTP Archaeology, which also set a clear structure for students to follow. However, there is little pretence that PATOIS could be used to replace or supplant other forms of teaching. Rather than presenting a complete curriculum, PATOIS is designed to support and extend other established means of learning. For example, one tutor set an essay question based on PATOIS, while other sessions were supported by lectures and seminars. This has proven something of a success, since students are more inclined to understand or be engaged with the material if they can see the place of the tutorials within wider study, and in particular if there is a cross-over between the tutorials and assessment. It is hard to imagine that computer-based learning would ever be able to supplant other teaching methods in archaeology, even for a topic as inherently IT-based as this. PATOIS is also open-ended. It promotes other forms of engagement, in particular the use of online resources, but for activities and projects driven by others, such as student dissertations or research projects.
The challenge of open-access Internet learning materials is how to take best advantage of the unexpected contexts of learning. Many will recognise and welcome the suggestion that resources need to be embedded with other forms of engagement, such as lectures, seminars, essays or tutorials. However, there is a danger that in emphasising the broader context, other less structured encounters are dismissed. That is certainly not the intention. The challenge of exploiting the multitude of possible contexts of learning is in part a challenge to balance the expectations of formal curricula - with the volatility of none. Use of PATOIS does not require a degree programme nor does it presume any.
PATOIS has benefited greatly from detailed, and at times invasive, evaluation and review. This review has been extensive, in part to cope with the unusually wide remit set for a national service that attempts to support students in many different courses and in a variety of settings. As the target audience increases, so the amount and importance of evaluation grows. One of the striking features of this peer-review process has been the consistently high quality of review that has been received, and the willingness of staff to participate in that review. The structure of peer-review for PATOIS has been more circuitous than would be the case for journal articles; every effort has been made to involve partners throughout the whole development of the tutorials, so the comments received have tended to require minor revisions only. Although this process of peer-review is not yet fully complete, it has worked well throughout. It would have been frustrating to spend several months working on a section in relative isolation and then be told that the whole effort was in vain or somehow misconceived. Not only has the peer-review guided the structure of the modules, but a welcome side effect has been to begin early dissemination, and also to ensure that expectations do not go ahead of realities. The programme of evaluation has been a real benefit to the team, and can be strongly recommended to others engaging in similar activities in the future.
Readers should be aware that the PATOIS project is not finished; two of the tutorials are still in their evaluation phase, and two of them are being re-furbished in the light of recent changes to the ADS catalogue. In truth, tutorials like this are never really finished - there are always more examples or themes to introduce or extend. It is thus too early to test whether the project has had any long-term success. However, it is possible to outline some measures of success by which it will be judged in the longer term.
It is hard and perhaps rather invidious to seek quantitative measures of success. PATOIS is about using digital resources, particularly those available from the ADS. If successful, PATOIS should increase the use of ADS resources but it will be difficult to isolate the impact of PATOIS within such a growth.
An alternative measure may be to identify the contexts of use and the speed with which students are able to find resources, as well as the confidence with which they approach issues like standards, archives, electronic publishing and so forth. Such broad cultural measures are much harder to capture than simple quantitative ones, but are much more useful in the long term. Measuring cultural change in the research community and finding the causes of that change requires ethnographic and sociological study, which can be informative but time consuming. PATOIS is hardly in a position to drive such far-reaching changes, but it may be possible to identify occasions when it has contributed. In the same way, it would be interesting to study the impact of TLTP as a hidden driver for IT skills across the archaeological community. Students who used TLTP had no choice but to come to terms with a computer, at a time when they were still viewed with some trepidation. The circumstances of PATOIS are different, but these sorts of long-term evaluation of cultural change within archaeology would be very welcome.
The best measure of success may be an increase in the number of institutions adopting PATOIS in their courses. If in a few years time the modules are still being used in a cross-section of higher education, then this will be deeply gratifying. In a strange way, the converse may also be true. It may well be that PATOIS acts as a stimulus to other - perhaps better - online teaching resources which either supplant PATOIS completely or revise it so much that it is no longer recognisable.
Last updated: Mon Sep 23 2002
© Author(s). Content published prior to 2013 is not covered by CC-BY licence and requests for reproduction should usually go to the copyright holder (in most cases, the author(s)). For citation / fair-dealing purposes, please attribute the author(s), the title of the work, the Internet Archaeology journal and the relevant URL/DOI.