University Assistance System HAnS

What does HAnS actually do?

University Assistant System (Hochschul-Assistenz-System, or simply HAnS) is a multifunctional learning platform that is meant to support students by enabling a much more efficient interaction with learning materials. It is still being actively developed, but it already offers several key benefits.

Enhanced lecture videos

HAnS platform allows teachers to upload their videos, just like any other, but using fine-tuned machine learning, HAnS makes these recordings much more useful:

• Automatic subtitle generation. No more being hindered by a bad microphone or a quite speaker.
• Lecture summarization. An automatic text summary of the lecture content allows judging at a glance what you should expect from the video and whether it contains the information you need
• Automatic table of content. A timestamped list of topics the lecture covers allows quick and efficient navigation, as well as an overview of the lecture logic. 
• Intelligent search. You remember that somewhere in the 3-hour lecture the professor mentioned something about a particular topic? No problem – you will be pointed to the exact timestamps where this topic was covered, with brief transcript extract, so you can understand the context.
• Automated question generation. Student can ask for questions of different difficulty level, concerning either the lecture content in general or the topic associated with the current timestamp, and have them automatically generated, allowing to quickly test whether or not they have understood the key concepts.
• Specialized chatbot. Specifically designed for the task and supported by a robust knowledge base, this chatbot gives student a personal tutor, ready to answer any questions they might have throughout the lecture.

And the most important thing is – through machine learning, the whole process is automated and can be performed for any uploaded lecture.
 

Content recommendation system

While still under development, this system already keeps track of what learning activities the student took part in, to recommend related lectures and other learning materials. This ensures that the student never gets stuck on their self-guided learning journey and can always ask for advice on what to tackle next. 

How to use HAnS?

Interested in trying the system out? Whether you are a student or a teacher, here we offer detailed instructions on how you should use the platform and how you can incorporate it into your teaching or learning workflow.

• Instructions for Students
• Instructions for Teachers
   

How does HAnS work?

While we can’t reveal all technical details, we’re happy to provide a general overview of what the system actually does:

• When the lecture is uploaded, a timestamped transcript is generated for it, using voice recognition. This transcript then serves as an input for several systems.
   
• Visual LLM processing combines the timestamped transcript with the lecture slides, creating a context, which defines the information the chatbot provides.
   
• Topic segmentation separates the transcript into several key segments, based on the topic, in order to enable efficient context selection for the chatbot answer generation. The table of contents is created similarly, but utilizes a separate system, as topic segmentation for the student and for the chatbot require different degree of detail.
   
• LLM summarization system uses the transcript to present a brief lecture summary, enabling quick initial navigation, while also using it to enable chatbot navigation of this information.
   
• Finally, embedding generation and LLM summarization are used to create question. Depending on the student’s needs, the embedded context which the questions are generated from, changes. For general lecture questions there is no focus on any specific embedded segment in particular. For timestamp-specific questions, the current timestamp the student is on is utilized as one of the inputs, ensuring that the context for question generation is tied to a specific segment of the transcript and, more importantly, the topic that segment is related to.
   

The whole system is computationally intensive, and, considering that multiple students and teachers can be using it at once, efficient resource allocation is extremely important, both to ensure all the system segments are accessible, and to make the system as energy and resource efficient as possible. This is achieved using Scalable Engine for Reasoning, Analysis, and Prediction with Hosting (SERAPH), which dynamically allocates all the tasks described above to a set of available hardware. If you want to learn more about how it does this – take a look at the relevant papers section below! 
 

About the project

HAnS is a joint project with 40 total contributors from 8 universities and 3 institutes across Germany. THI’s work focuses first of all on the "User perspective and acceptance" work package, ensuring that the interaction between the users and the underlying systems goes as smooth and as efficiently as possible. On top of that we are also involved in the technical development and are responsible for integrating HAnS into THI Moodle. 

Publications