Developing a digital learning environment: an evaluation of design and implementation processes

ABSTRACT

The Alexandria Digital Earth Prototype (ADEPT) Project (1999--2004) builds upon the Alexandria Digital Library Project (1994--1999) to add functions and services for undergraduate teaching to a digital library of geospatial resources. The 'Digital Learning Environment' (DLE) services are being developed and evaluated iteratively over the course of this research project. In the 2002--2003 academic year, the DLE was implemented during the fall and spring terms in undergraduate geography courses at the University of California, Santa Barbara (UCSB). Evaluation of the fall term implementation identified design issues of time and complexity for creating and organizing course domain knowledge. The spring term implementation added new services to integrate course content into class presentation formats. The implementation was evaluated via interviews with the course instructor, development staff, and students, and by observations (in person and videotaped) of the course. Results indicated that usability and functionality for the instructor had increased between the two course offerings Students found classroom presentations to be useful for understanding concepts, and Web access to the presentations useful for study and review. Assessments of student learning suggest modest improvements over time Developers are now applying lessons learned during these implementations to improve the system for subsequent implementation in the 2003--2004 academic year.

REFERENCES

• 1: Apple, M. Teaching and technology: The hidden effects of computers on teaching and students In Beyer, M A L ed The Curriculum: Problems, Politics, and Possibilities New York: State University of New York Press, 1998, 314--336.
• 2: Ann Peterson Bishop , Nancy A. VanHouse , Barbara P. Buttenfield, Digital Library Use: Social Practice in Design and Evaluation, MIT Press, Cambridge, MA, 2003
• 3: Borgman, C. L. Smart, L. J. Millwood, K. A. Finley, J. R. Champeny, L. Gilliland-Swetland, A. J. and Leazer, G. L. Comparing Faculty Information Seeking in Teaching and Research: Implications for the Design of Digital Libraries Journal of the American Society for Information Science & Technology, in press.
• 4: Christine L. Borgman , Gregory H. Leazer , Anne J. Gilliland-Swetland , Rich Gazan, Iterative Design and Evaluation of a Geographic Digital Library for University Students: A Case Study of the Alexandria Digital Earth Prototype (ADEPT), Proceedings of the 5th European Conference on Research and Advanced Technology for Digital Libraries, p.390-401, September 04-09, 2001
• 5: Borgman, C. L. Gilliland-Swetland, A. J. Leazer, G. H. Mayer, R. Gwynn, D. Gazan, R. and Mautone, P. Evaluating digital libraries for teaching and learning in undergraduate education: a case study of the Alexandria Digital Earth Prototype (ADEPT) Library Trends 49(2) (2000), 228--250.
• 6: Bowker, G. C. and Star, S. L. Social theoretical issues in the design of collaboratories: customized software for community support versus large-scale infrastructure In Olsen, G. M. Malone, T. W. and Smith, J. B. eds. Coordination Theory and Collaboration Technology Mahwah, NJ: Lawrence Erlbaum Associates, Inc 2001, 713--738.
• 7: Nicholas C. Burbules , Thomas Callister, Watch It: The Risks and Promises of Information Technologies for Education, Westview Press, Inc., Boulder, CO, 1999
• 8: Chen, Q, and Sharma, V. Human factors in interface design: an analytical survey and perspective In Szewczak, E. J. and Snodgrass, C. R. eds. Human Factors in Information Systems Hershey, PA: IRM Press, 2002, 45--54.
• 9: Larry L. Constantine, Beyond chaos: the expert edge in managing software development, ACM Press, New York, NY, 2001
• 10: Gregory Crane , Robert F. Chavez , Anne Mahoney , Thomas L. Milbank , Jeffrey A. Rydberg-Cox , David A. Smith , Clifford E. Wulfman, Drudgery and deep thought, Communications of the ACM, v.44 n.5, p.34-40, May 2001 [doi>10.1145/374308.374333]
• 11: Cuban, L. Teachers and Machines Cambridge, MA: MIT University Press, 1986.
• 12: Dewald, N H and St Clair, G. S. Active and collaborative learning in online courses. In Snyder, C A and Fox, J W eds Libraries and other Academic Support Services for Distance Learning Greenwich, Conn: JAI Press, 1997.
• 13: Kweku Ewusi-Mensah, Software Development Failures, MIT Press, Cambridge, MA, 2003
• 14: Gilliland-Swetland, A. J. Evaluation design for large-scale, collaborative online archives: Interim report of the Online Archive of California Evaluation Project Arch Museum Informatics 12 (1998), 177--203.
• 15: Hay, K. and Barab, S. Constructivism in practice The Journal of the Learning Sciences 10(3) (2001), 281--322.
• 16: Linda L. Hill , Larry Carver , Ron Dolin , Terence R. Smith , James Frew , Mary-Anna Rae, Alexandria digital library: user evaluation studies and system design, Journal of the American Society for Information Science, v.51 n.3, p.246-259, Feb. 2000 [doi>10.1002/(SICI)1097-4571(2000)51:3<246::AID-ASI4>3.3.CO;2-Y]
• 17: Greg Janée , James Frew, The ADEPT digital library architecture, Proceedings of the 2nd ACM/IEEE-CS joint conference on Digital libraries, July 14-18, 2002, Portland, Oregon, USA [doi>10.1145/544220.544306]
• 18: Kling, Rob What is social informatics and why does it matter? D-Lib Magazine 5(1) (1999) {Retrieved October 17, 2003 from TUhttp://www dlib org/dlib/january99/kling/01kling htmlUT}.
• 19: Marchionini, G. Evaluating digital libraries: A longitudinal and multifaceted view Library Trends 49(2) (2001), 302--333.
• 20: Byron Marshall , Yiwen Zhang , Hsinchun Chen , Ann Lally , Rao Shen , Edward Fox , Lillian N. Cassel, Convergence of knowledge management and E-learning: the GetSmart experience, Proceedings of the 3rd ACM/IEEE-CS joint conference on Digital libraries, May 27-31, 2003, Houston, Texas
• 21: Martin, F. G. Ideal and real systems: a study of notions of control in undergraduates who design robots in Kafai, Y and Resnik, M eds Constructionism in Practice: Designing, Thinking and Learning in a Digital World Mahwah, NJ: Lawrence Earlbaum Associates, 1996, 297--322.
• 22: Richard E. Mayer, Multimedia Learning, Cambridge University Press, New York, NY, 2001
• 23: Mayer, R. E. and Mautone, P. and Prothero, W. Pictorial aids for learning by doing in a multimedia geology simulation game Journal of Educational Psychology 94(1) (2002), 171--185.
• 24: Steve McConnell, Professional Software Development: Shorter Schedules, Higher Quality Products, More Successful Projects, Enhanced Careers, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 2003
• 25: Modell, H. I. and Michael, J. A. eds. Promoting Active Learning in the Lfe Science Classroom New York, NY: New York Academy of Sciences, 1993.
• 26: Owen, J. M. Calnin, G. T. and Lambert F. C. Evaluation of information technology (Chapter 6) In Altschuld, J. W. and Kumar, D. D. eds. Evaluation of Science and Technology Education at the Dawn of a New Millenium New York: Kluwer Academic/Plenum Publishers, 2002, 133--164.
• 27: Sandoval, W. A., and Reiser, B. J., Explanation--driven inquiry: integrating conceptual and epistemic supports for science inquiry Science Education 87 (2003), 1--29.
• 28: Smith, T., R. Ancona, D. Buchel, O. Freeston, M. Heller, W. Nottrott, R. Tierney, T and Ushakov, A The ADEPT concept-based digital learning environment. In Koch, T and Solvberg, IT eds. Research and Advanced Technology for Digital Libraries, 7th European Conference (ECDL 2003) (Trondheim, Norway, August 17--22, 2003) Berlin: Springer, 2003, 300--312.
• 29: Smith, T., R. Zeng, M L and the ADEPT Knowledge Team Structured models of scientific concepts for organizing, accessing, and using learning materials. In López-Huertas, MJ ed Challenges in Knowledge Representation and Organization for the 21st century: Integration of Knowledge Across Boundaries, Seventh International ISKO Conference (Granada, Spain, July 10--13, 2002) Advances in Knowledge Organization 8 Würzburg: Ergon, 2002, 232--239.
• 30: Soergel, D. An information science manifesto (remarks at the acceptance of the Award of Merit, American Society of Information Science, Washington, DC, November 5, 1997) Bulletin of the American Society for Information Science 24(2) (December 1997/January 1998), 10--12.
• 31: Weedman, J. The structure of incentive: design and client roles in application--oriented research Science, Technology & Human Values 23(3) (1998), 315--345 {1--20}. {Retrieved March 6, 2004 from http://infotrac galegroup.com/}.