# PAIL: Personalising Assessment for Individualised Learning PAIL is a tool designed to automate the generation and personalising of assessments to assist with individualised learning. ## Background During the covid-19 pandemic, delivery of almost all courses moved online; now, universities commonly offer flexible/hybrid models of teaching and learning ([Crick et al, 2021](https://doi.org/10.1109/FIE44824.2020.9274033)). With the rise of online and/or hybrid teaching ([Turnbull et al, 2021](https://doi.org/10.1007/s10639-021-10633-w)), there has been renewed interest in assessment – its authenticity, the value of examinations, and the corresponding dependence on online assessments and electronic invigilators ([Raptana et al, 2021](https://doi.org/10.1007/s42438-021-00249-1)). The latter has been reported to be vulnerable to problems related to online assessment platforms and associated electronic invigilation tools, and there have been numerous reports of increasing incidence of academic integrity violations, particularly plagiarism and collusion ([Simon et al, 2022](http://leemans.ch/publications/papers/ace2022simon.pdf)). Using assessment-as-learning ([Yan et al, 2021](https://books.google.com.au/books?id=xl5CEAAAQBAJ&lpg=PP1&ots=7dk5VI-Mp8&lr&pg=PP1#v=onepage&q&f=false)) in the context of ICT, and especially cybersecurity ([Leszczyna, 2021](https://doi.org/10.1016/j.cose.2021.102376)), an assignment looks the same for all students but each assessment presents different input data to students. Students are encouraged to work together to solve their assignment questions and to learn from one another, but they do the assignments on their own since the input data is tailored for each student. While this approach brings assessment-as-learning to the fore, it also has been found to be effective in reducing the incidence of plagiarism and collusion in assessments ([Bretag et al, 2019](https://doi.org/10.1080/02602938.2018.1527892)). This approach has the dual benefits of enhancing peer learning and peer feedback through assessment-as-learning while also reducing academic integrity issues associated with online platforms. Personalising of assessment ([Jones and McLean, 2018](https://link.springer.com/content/pdf/10.1007/978-981-10-7930-6.pdf)) is the act of formalising an assessment to bring different communities together to enrich and broaden the learner’s experience, through empowering learners to take more control over what and how they learn. This is different from ‘personalised assessment’, which is instruction that is paced to learning needs, tailored to learning preferences, and tailored to the specific interests of different learners. In an environment that is fully personalised, the learning objectives and content, as well as the method and pace, may all vary. Personalised assessment relies heavily on the notion of individualisation and differentiation, which often risks leaving the learner to work alone on an individual task, something that may isolate students. This is in contrast to the notion of personalising of assessment, which is far more focused on the social aspects of learning through collaboration with peers and the wider community for an assessment task. ## Description PAIL provides a framework to allow the automatic generation of personalising assessments by allowing the instructor to provide a program that will generate individual files for that learner and then present them with a personalising rendered [Markdown](https://en.wikipedia.org/wiki/Markdown) file(s). The system uses [Learning Tools Interoperability (LTI)](https://en.wikipedia.org/wiki/Learning_Tools_Interoperability) through a [Learning Management System (LMS)](https://en.wikipedia.org/wiki/Learning_management_system) to determine the directory where the personalising files should be stored for each individual learner. ## Setup PAIL is a PHP wrapper that uses a script to automatically create personalising assessments for users, and then display an associated Markdown file. To set up, you should upload the PAIL files to a PHP server and modify ```settings.php``` to give the following details: | Setting | Description | | ------- | ----------- | | ```$pail_oauth_consumer_secret``` | The LTI consumer secret that should be used by the LMS | ```$pail_title``` | The title for the page | ```$pail_description``` | A short description of the assessment | ```$pail_generate_command``` | The command used to generate the personalised assessment for the user. It will be passed the directory in which to store any generated files as its last command line argument. There are no checks that this program does only store files in the specified directory, so you should ensure you trust that this command will do the right thing. | ```$pail_contact``` | The contact email people should be directed to if there are any issues | ```$pail_reset``` | If false (the default), files are only generated the first time the learner visits the LTI tool (and simply displayed on subsequent visits). If true, the files are generated each time the learner visits. | ```$pail_generated_dir``` | The subdirectory where files will be generated (defaults to "generated") - note that you will need to change this in ```generate.sh``` if you change it here | ```$pail_context_id``` | The key to use to get the context ID from the LTI context (defaults to "context_id", which should always be appropriate) | ```$pail_user_id``` | The key to use to get the user ID from the LTI context (defaults to "user_id". To get student number at UNE, use "lis_person_sourcedid" instead) | ```$pail_debug``` | Whether to output debug information with the page (defaults to false) You can then set up an LTI tool in your LMS that points to the base URL where you have PAIL set up. PAIL will then automatically generate and render the Markdown file specified by the ```display``` request parameter (or ```display.md``` if no such parameter exists) for each individual learner. For example, if set up at https://example.edu/pail/, https://example.edu/pail/ would display ```display.md``` and https://example.edu/pail/?display=example.md would display ```example.md```. ### Generate command The generate command is passed the directory where it should generate files as its final command line parameter. Note there are very few checks that this program behaves "nicely" (e.g., only generates files in the correct directory), so you should trust that this program works correctly. The file rendered by PAIL is ```generated/CONTEXT/USER/DISPLAY```, where CONTEXT is the ```context_id``` obtained from the LMS via LTI, USER is the ```user_id``` obtained from the LMS by LTI, and DISPLAY is the value of the ```display``` request parameter (or ```display.md``` if no such parameter is set). ## Authors and acknowledgment This project is initally supported by the [Australian Council of Deans of Information & Communications Technology (ACDICT)](https://acdict.edu.au/) under their [ALTA small grants scheme 2022](https://acdict.edu.au/alta-small-grants-scheme-2022/) program for the project "Technical and Educational Guidelines for Personalising Assessments through Assessment-as-Learning in Cybersecurity". Project participants are: * Dr Amin Sakzad (Monash University, Australia) * Dr David Paul (University of New England, Australia) * Prof Judithe Sheard (Monash University, Australia) * Prof Ljiljana Brankovic (University of New England, Australia) * Dr Matthew P. Skerritt (Royal Melbourne Institute of Technology, Australia) * Dr Nan Li (University of Newcastle, Australia) * A/Prof William Billingsley (University of New England, Australia) * Dr Simon ## License This project is released under the [MIT License](https://mit-license.org/) (see LICENSE)