Correlation between the Practical Aspect of the Course and the E-Learning Progress

Abstract

The aim of the study was to analyze the correlation between the students’ involvement and their progress within the online component of the blended learning model during the theoretical and practical courses. The authors have applied data mining Moodle from learning portfolios of 1500 students (N = 1500). Comparative analysis of the courses under review showed significant differences in the behavior of the same students. For the theoretical course students spent less time than for a practice-oriented one. Students’ progress in the form of points and demand for non-binding elements of the course differed significantly in favor of the practical course. Based on the data obtained, the authors concluded that an important parameter that influences the behavior of students and their educational progress in general is precisely the focus on practice. Thus, the predominance of the online component in blended learning is appropriate for practical courses. Within the theoretical and general education courses, the predominance of face-to-face learning can positively affect the educational process and results of learning.

1. Introduction

Information and communication technologies, penetrating into all spheres of human life, greatly change the image of modern education [1,2,3,4,5,6]. The evolving e-learning system over the world requires consideration of modern education that takes into account both traditional factors and the influence of the latest technologies. Researchers connect learning progress with a variety of factors. In the most general form, these factors can be divided into course-related (internal) and student-related (personal) aspects [7].

A number of internal factors will be the same for traditional, blended and e-learning courses. However, implementation of education by the means of information and communication technologies creates a new environment for learning process. Technologies turn out to be an important factor influencing learning [8,9,10], even interface design [11]. In this context the role of social interaction in the e-learning process is increasing that is connected on the one hand with the teacher’s feedback and on the other hand with the students’ communication and collaborativeness with classmates [12,13,14,15]. That is why we should note the influence of connectedness, which can be supported by social networks [16]. We follow the theory of transactional distance developed by Moore, combining the main success factors related to the online course: Structure, dialogue and autonomy [17]. The community of inquiry (CoI) framework proposes to consider the three essential elements: Cognitive presence, social presence and teaching presence [18,19]. A number of studies contain data from sociological studies of the students and teachers’ opinions about the elements of an online course ideal for teaching, course parameters measuring [20,21,22], for example, in the context of cognitive investment, socio-emotional and behavioral engagement [23].

In the on-line environment, the degree of student’s freedom in the learning process becomes regulated, therefore the role of self-regulated learning [24] is studied as self-directed e-learning [25], as well as options for combining e-learning and face-to-face instruction in blending learning [26,27].

Among personal factors psychological factors are investigated: Self-efficacy [28], emotional intelligence, componential intelligence, experimental intelligence and contextual intelligence [29], positive self-concept, realistic self-appraisal, preference for long-term goals, leadership experience, community involvement and knowledge acquired by individual’s preferred learning style [30], higher level of independence in the learning process [31], attitudes of students towards technologies [32] and the demographic factor [33]. The ability to manage your time becomes one of the most important skills of a student. A lot of research is devoted to the impact of procrastination on successfulness of online learning [34,35]. A number of studies link the motivation and success of students [36,37,38,39]. For online learning, a self-determination theory is used as the methodology that connects intrinsic and extrinsic motivation [40].

There are effective approaches to the methods of identifying factors for online learning progress. One approach is oriented to study those who have not completed the course online. The classical Tinto’s model of institutional departure (based on the degree of academic integration and social integration) [41,42] subjected to criticism due to the lack of a relevant rating system for these factors evaluation [43,44] and turns out to be very limited in the electronic environment. Therefore, researchers are studying the reasons for leaving the course by conducting surveys of this category of students and analyzing specific personal traits [33,45].

Another approach lies in evaluating the learning portfolios of students recorded in a learning management system (LMS) using the data retrieved from system logs or a database [46,47,48,49,50]. The authors seek to identify those factors that, being available for computer extraction and processing, are predictors of student’s performance. Although identifying factors in the behavior of students on an online course that correlate with the assessment is extremely interesting and necessary for students who are lagging behind in the online learning. This task is at the interface of pedagogy and computer science [48,51,52] and it is clear that the online behavior itself is a big part of the learning process. For example, there are such findings from data mining Moodle that the number of quizzes passed predicts the final mark [51] or that low level of contributions to the course learning activities leads to failure [53]. For instance, with the help of Rip, the Ripper implementation in Weka we may predict that with a low initial level of knowledge and minimal time spent, the student will receive a low mark [54] (p.624).

Modern technologies allow you to have a complete understanding of the behavior of students during the course, but so far, this dataset has limited application. The grade for the e-course itself is one of the indicators of students’ behavior.

Figure 1 shows how the online educational process can be analyzed. Arrows show the elements’ correlations, which are most often analyzed in the literature. In the process of e-learning, we observe the interaction between students and the e-learning environment. Initially, students were most often analyzed to assess online education. These could be personal factors, students’ evaluation of the course, as well as their own behavior during the course (the green color refers directly to the existing educational environment and behavior within it, and the yellow one relates to the course design). Now for analysis, we can use data describing students’ behavior, obtained directly from the educational environment. Curves in the oval symbolically demonstrate a variety of behavioral strategies during the course. We marked variants with poor results or fail. At the same time, it is shown that the assessment indicates only certain behavior parameters during the course.

Summarizing the main areas of research to analyze the effectiveness of e-education, the following areas can be identified:

-

Correlation between personal factors and behavior/grades during the course;

-

Analysis of students’ ideas about their behavior during the course (for example, the reasons for departure of the course);

-

Analysis of the persons’ comprehension of the course who are involved in learning (sociological research on the course assessment, the ideal course);

-

Analysis of the correlation between internal factors of the course with the students’ behavior or students’ grades or their evaluation of the course (for example, studying different options for creating communication during the course, or comparing blended and online learning with educational outcomes and grades);

-

Students’ behavior during the course and grades.

At the same time, due to the emphasis on electronic specificity, among the internal factors the features of the course itself are not taken into account. Despite the similar structure of the presentation of educational information in each online environment, various subjects cannot be considered as similar. However, researchers most often analyze performance in individual subjects (autonomously or in comparison with on-line, blended and face to face) or on an array of courses, without highlighting the difference in the subject. The influence of the discipline features is considered only in several research, so in a recent study [55] an attempt was made to compare the communicative behavior between a humanitarian and technical course. Though in [56] it is indicated that the course subjects affect student’s engagement, but it does not even reveal which courses were specifically studied.

The present study aimed to compare online behavior and student success in courses that differ in their degree of practice orientation. In this paper, theoretical course study implies the acquisition of new knowledge and skills of intellectual work. A practical course is connected with the additional possibility of direct implementation of the knowledge in order to achieve immediate practical results. In this case, we do not mean implementation of "practical" tasks given by the teacher. However it implies that the learning materials are used as a mean of collecting, analyzing and presenting information [57] (p.9) or for independent work (for example in laboratories). Such a blended learning scheme—a combination of online theory and direct practice is used in various fields, for example, in medicine [58,59]. The modern online environment allows practicing direct practical skills in the form of modeling various professional activities from surgery to aviation [60,61,62,63], which can be understood as "the most practical course". However, such courses are too specific and cannot be compared with theoretical ones.

2. Materials and Methods

The data of the students involved in the “History” and “Project activity foundations” (PAF) courses was analyzed in the study during the academic year 2017–2018. To assess the involvement of students of the St. Petersburg Polytechnic University in this study, we used data mining Moodle from learning portfolios of students.

Before starting the study, St. Petersburg Polytechnic University students agreed to the processing of personal data. This also applied to the “History” and “Project activity foundations”. The processing of personal data was limited by the achievement of specific, predetermined research goals. Access to information was presented only in a generalized and non-personalized form.

The study was based on data from the portfolio of the first-year undergraduate students who had taken "History" and "Project activity foundations" courses at St. Petersburg Polytechnic University from September 2017 to June 2018. Course duration did not exceed one semester, so the "History" course was held in the first semester of the 2017–2018 academic year, and the "Project activity foundations" course in the second semester. In accordance with the course requirements outlined in the curriculum, students were required to attend online classes and watch online videos during certain time periods, with user authentication in the LMS. For each student, the following actions were recorded in the LMS databases, including entering and exiting the e-learning system, opening events of the course glossary, and the effectiveness of the test final assignments. All student activities were stored in log file format, meaning that a record was generated in the LMS database as an event occurred. Therefore, a number of data preprocessing tasks must be performed, such as session identification, data integration and data aggregation.

The sample set was formed on the basis of the quota sample (N = 1500 respondents). The sample of the research was proportional to the institutes and specialties represented at the university; the total amount was 4000 students. Gender composition of respondents: Men = 68%, women = 32%, age structure: 18–19 years = 88%, 20–21 years = 12% (

Table 1. Demographic profiles.

Profiles	Description	Percentage (%)
Gender (Male/Female)	Male	68
	Female	32
Age	18–19	88
	20–21	12

).

The choice of St. Petersburg Polytechnic University was due to the fact that this university was one of the first in the Russian Federation where we began to implement blended learning. This was preceded by the entry of St. Petersburg Polytechnic University in December 2014 as one of the main performers into the federal project of the Ministry of Education and Science of Russia to create a National Open Education Platform. The main goal of this process is to improve the quality, relevance and accessibility of Russian education through the wide and effective use of online courses of leading lecturers.

Currently, e-learning has been introduced into the educational programs of St. Petersburg Polytechnic University at all levels of training. In 2019, 52% of undergraduate and graduate students were involved in e-learning. It should be noted that all students of the first and second year undergraduate courses, regardless of the institute and educational program, took at least five distance courses during the academic year.

The study of the students’ progress at the online courses of St. Petersburg Polytechnic University focused on the problem of the student’s involvement in distance learning courses of various practical orientations that raised a number of research questions. On the one hand, it was necessary to understand the sameness of students’ behavior during the "History" and "Project activity foundations" courses. On the other hand, it was necessary to determine how the practical component of the courses could correlate with the views of students on the online format of higher education, their demands and expectations, which are largely derived from the ideas of their future career and the role of higher education in their professional development.

The purpose of the "History" course was to form a comprehensive view of students on the cultural and historical uniqueness of Russia, its place in the world and European civilization, to give systematized knowledge about the basic laws and features of the global historical process, emphasizing the history of Russia, to introduce into the circle of historical problems connected with the area of future professional activity, to develop skills for obtaining, analyzing and summarizing historical information. The course program consisted of three modules: “History as a science, theoretical foundations of discipline (methodology, historiography, sources)”, “History of Russia and the world before the beginning of the Newest time” and “Newest and modern history”. Weekly classes included topical video lectures (duration up to 15 minutes); reading electronic notes; doing various tasks with automated verification of results (after each module) and analyzing materials for self-study. Optional aspect: Discussion of learning outcomes on the forum. At the end of each module students were tested to assess their knowledge. The course ends with a final test and on the basis of these data a total final grade for the course was formed. In the 2017–2018 academic year, 4000 students completed the course under the guidance of 10 university professors. The course lasts 16 weeks.

The course “Project activity foundations” gives an overview of how the project activities are carried out, as well as step-by-step guidance in applying the project approach. It provides the answers to such questions: “Why the result of the project was not what was planned?”, “How to plan results before they appear?”, “What to do, when there is a discord in the team, and does the discord mean that the team has no future?”, “What should you do to complete the project in time according to budget and content?”, “What should you do if something goes wrong?”, etc.

This course is practical and is used to improve the effectiveness of learning materials study during the “Project activity foundations” course. The students are recommended to choose a simple project before starting the course: Some aspects of everyday life or professional activity (arrangement of cycle parking, website development, etc.) and during the course apply tools to the selected project.

As a result of the course, the student develops such competencies as the ability to search, critically analyze and synthesize information, the ability to define a range of tasks within the framework of a goal and choose the best ways to solve them, to apply a systematic approach to solve the tasks, basing on current legal norms and available resources and restrictions.

Weekly classes included topical video lectures (duration up to 15 minutes); reading electronic notes; doing various tasks with automated check of results and analyzing materials for self-study. Optional aspect: Practical work on project, which, if desired, can be discussed on the face-to-face classes. The program of the course consisted of 11 topics: A general idea of the project activity, the formation of a project team, communication in a team, project idea development, presentation of an idea, project life cycle, development of performance requirements, project work planning, budget and project risks, project management methods and objectives at the implementation and final stages. The discussion of learning also took place on the course pages on social networks, where students could not only ask questions to teachers, but also shared photo/video information about the implementation of projects after the course. In the 2017–2018 academic year, 4000 students were divided into teams of four to eight people and implemented 512 projects, under the supervision of 93 university lecturers. The course lasts 14 weeks.

The "History" and "Project activity foundations" courses were implemented by the blended learning model. Blended learning is an educational approach that combines learning with the participation of a teacher and online learning, but the latter should not exceed 80%. We compare these courses by the number of hours given for the various activities of the curriculum.

Contact hours with a teacher during these courses did not differ very significantly, six hours for practice classes, 1.5 hour for lectures in the “Project activity foundations" course and three hours for the “History” course. However, the student’s autonomous work was different, since one of the courses was practice-oriented and required 50.25 hours (

Table 2. Distribution of hours in the curriculum.

	History	Project Activity Foundations
Lectures	3	1.5
Practice classes	6	6
On-line lectures	6	10.5
On-line practice classes	12	6
Autonomous students’ activities	20.25	50.25

).

The authors drew attention to the fact that the hours allotted for online activities were presented according to the curriculum of the discipline. It is the recommended activity time for a student. It is not obligatory, but an approximate, as each student, depending on various factors, spends a different number of hours for lectures studying and practical tasks online.

We should note that communication with a teacher and fellow students through the forum was optional and this affects the activity of students. The problems discussed in this way were mainly technical or organizational aspects. The volume of communication on the forums within the two courses was not significantly different.

In order to assess the quality of student achievements in the distance "History" and "Project activity foundations" courses a point system was used for intermediate certification (credit). The amount of points received by the student for the semester was recalculated into the grade for the discipline according to the ECTS (European Credit Transfer and Accumulation System) scale.

A prerequisite for the evaluation of student learning outcomes was the fulfillment of all compulsory types of planned educational work in a distance course discipline. Students, who scored 60 or more points during the current and mid-term certification and completed all the required types of planned training sessions within a discipline with intermediate control in the form of a test, automatically received a credit in this discipline (

Table 3. Grade scale.

Grade in Score	Grade in National Scale	Grade in ECTS Scale
90–100	Excellent	А	Excellent (excellent assignment with only insignificant amount of errors)
82–89	Good	B	Very good (above average with a few mistakes)
75–81	Good	С	Good (overall correct assignment with a certain number of significant errors)
67–74	Satisfactory	D	Satisfactory (not bad, but with a significant number of mistakes)
60–66	Satisfactory	E	Adequate (fulfillment satisfies minimum criteria)
35–59	Unsatisfactory	FX	Insufficient (possible test retake)
1–34	Unsatisfactory	F	Insufficient (with obligatory repeated course)

).

The study applies the general scientific group of formal-logical methods used at both the empirical and theoretical levels, such as analysis, synthesis, abstraction, as well as the system approach. The system approach allowed revealing features of practice-oriented courses comprehension by students in the process of blended learning, to identify the relationship between student involvement and course progress, course duration, as well as appeal to additional information. Moreover, system approach principles were used in analyzing the forums of the “History” and “Project activity foundations” courses on the LMS platform, communities of the “Project activity foundations” course in social networks and the programs of the studied disciplines. We also used such methods of scientific research as structural analysis, design of logic circuits and graphical interpretation of empirical data. In this study, we also used an interdisciplinary approach, which provided consideration of students’ involvement and progress in the perspective of the education sector problems based on the achievements of sociology and pedagogy.

3. Results

We considered the processed data from individual student profiles from the LMS platform with such indicators as the total number of hours per course, reference to the course glossary, as well as the progress rates, represented according to the ECTS scale.

Figure 2 shows a diagram with an hour’s data. We analyzed time spent by students for the online component of the courses (x-axis is the distribution of hours, y-axis is the number of students). The recommended course time in the online format is 18 hours (History) and 16.5 hours (Project activity foundations). We would compare it with the real time spent by students during the “History” and “Project activity foundations” courses. Most students spent no more than four hours on online education during the semester: 81% for the “History” course and 66% for the “Project activity foundations” course. Only 0.3% of students stuck to the recommended training time of the “History” course, and 1.2% of the “Project Activity foundations” course. The distribution of students by time spent on the course “History”: Up to one hour—14.4%, from one to two hours—28.3%, from two to four hours—38.6%, from four to six hours—12.5%, from six to eight hours—3.5%, from eight to 10 hours—1.1% and more than 10 hours—1.6%. “Project Activity foundations” course: Up to one hour—7.7%, from one to two hours—19.7%, from two to four hours—38.9%, from four to six hours—18.7%, from six up to eight hours—7%, from eight to 10 hours—3.7% more than 10 hours—4.3%.

At the same time, it is important to note that the distribution of “PAF” course hours in the column diagram was close to the normal distribution, in contrast to the “History” course (Figure 2). It indicates the difficulty of comprehension of the theoretical, general education course in this format and students’ attempts to ignore part of online educational activities, focusing only on the mandatory and verifiable parts of the course.

Figure 3 shows a chart with data describing the number of students that accessed the online glossary of the course during the learning (the x-axis is the distribution of the number of students’ access to glossary, the y-axis is the number of students). Appeal to the glossary is an important marker of student involvement in the course. The glossary in the distance part of the course is often a lexicon of terms in the given field of knowledge with the interpretation, sometimes translation of foreign expressions into Russian, comments and examples.

In the History course, 20.2% of students used it. 28.1% of students used the “PAF’ course. At the same time, a distinctive feature of the PAF course was more than 10 student referenced to the course. In the “History” course 9.2% of students and 9.4% of students of the “Project Activity foundations” course applied to the glossary at least once. At the same time, a distinctive feature of the course on project activities was the presence of students with a repeated and systematic number of requests to study this block. From two to 10 times this section was viewed by 10.8% of students, from 10 to 20 times 4.9%, from 20 to 30 times 1.7% and more than 30 times was 1.3%. In the “History” course, there were no references more than 30 times, from two to 10 times there was 9.7% of students, 10 to 20 times 0.9% and from 20 to 30 times 0.2% (Figure 4). It is worth paying attention to the fact that the glossary in the LMS platform is a convenient tool that can speed up the digestibility of the material. If the term described in this dictionary comes across in the text of the course material, it will automatically become a link. This link leads to the definition of the term. Thus, the student does not need to leaf through the entire course to find the right definition. In this sense, we supposed that the difference between the courses was primarily related to their specificity. Since the “History” course is a continuation of the school course, so most students understand the definitions found in it. On the contrary, the “Project Activity foundations” course includes new material for students of almost all specialties.

Figure 4 shows a chart with data describing students’ final progress (the x-axis is the distribution of points according to the ECTS scale, the y-axis is the number of students). The main conclusions about the productivity of the discipline study can be made from the final assessments of courses. The greatest number of A (excellent, assignment with only a few errors, from 90 to 100 points) was received by students of the “Project Activity Foundations” course (67.7%), although the results on this parameter did not give a big gap comparing with the “History” course (66.3%). In the “Project Activity foundations” course 13.9% of students and 7.5% of the “History” course had a B grade (very good results, above average with a few mistakes, from 82 to 89 points). There was a small difference in the number of students of the “Project Activity Foundations” course (5.3%) and “History” course (4.5%) who received C (good, overall correct assignment with a certain number of significant errors, from 75 to 81 points) grades. Grade D (satisfactory, not bad, but with a significant number of mistakes, from 67 to 74 points) and E (Adequate, fulfillment that satisfies the minimum criteria, from 60 to 66 points) also did not have large differences. “History” course: D—1.6% of students and E—1.1% of students. “Project Activity Foundations” course: D—2.5% of students and E—1.5% of students. Considering students who did not successfully complete the course, we saw a significant gap. These students did not score 60 points (summing up FX and F, i.e., unsatisfactory (with the possibility of retake or with a mandatory repeat course)): 9.1% of students of the “Project Activity foundations” course and 19% of the “History” course.

Considering students who did not successfully complete the course, we saw a significant point gap. Students who were unable to successfully complete the course and did not score 60 points were 9% of the “PAF” course and 19% of the “History” course (Figure 4).

4. Discussion

The learning management system allows an interested circle of users (university administration, course developers, teachers and tutors) to obtain monitoring tools for learning activities in the online environment that are in high demand today [8,13,25,30,32]. A variety of data describing students’ behavior in the electronic educational environment (for example, [56]) provides objective information about the features and dynamics of the educational process. For example, a set of comparable statistical reports and course evaluation reports provide information useful for optimizing course design. In addition, the use of these resources allows teachers to quickly edit and update learning materials. Thus, we could conclude that the data obtained could be used for a wide range of analysis, not limited to students’ estimates (as in [51,52,53,54]).

Comparative analysis of the electronic "History" and "Project activity foundations" courses showed significant differences in the behavior of the same students. In addition, it confirms the results [56], that course subjects have a significant impact on the students’ behavior online. Students spent significantly less time for the “History” course than for the “Project Activity foundations” course. Achievements in the form of points also differed in favor of the latter. Although we saw that the percentage difference of the students who successfully completed the course was insignificant (only 2%), whereas unsuccessful (not having a passing score) were 2.2 times more (more than 120%).

The hypothesis of the study was that a critical parameter affecting the behavior and success of the course is precisely the focus on practice. Obvious practical orientation of the PAF course creates a situation of demand for e-course information. The immediate benefits extracted and applied "here and now" contribute to greater self-control and engagement in the course. The purpose of the course, the study of which was not related to the practice, seems to be recognized by a number of students as the final tasks passing with the maximum saving of time spent.

E-learning turns out to be a test of a person for self-efficacy, maturity and other qualities of a formed personality. In addition, for unprepared students, internal factors related to the course will play an important role. Thus, it seems to us that the predominance of the online component was justified for practiced courses. For theoretical, general education courses, the predominance of online to face-to-face learning has a negative effect on the educational process and the learning outcomes of students, especially, with "a surface approach to learning" (in Biggs terminology) [64].

It is also worth paying attention to the time that students spend on the online part of the learning. Most of them spent less time than the recommended for the “History” and "Project activity foundations" courses. This undoubtedly identifies the problem and is of great interest for future research. It also points to the problem for course developers and teachers.

The limitations of the study were related to the possibility of unaccounted factors contributing to the difference in students’ behavior and results of learning. First of all, it is the personality of the teacher and his/her professionalism, both as being a lecturer at a distance course, as coming to an audience (for example, negatively thinks of online education), tutor support. We also note that the “History” course was held in the first semester, and the Fundamentals of Project Activity course was in the second semester. Thus students acquired some learning experience when they stepped to the second course, which could have a positive effect on further online education. An important factor is the communication between students and the teacher, but in both courses it was not sufficiently advanced, and was of a casual, uncontrollable nature. This situation did not meet the requirements of quality learning discussions [65]. At the forum, located directly in the LMS courses, student activity was rather low or absent. However, the “Fundamentals of Project Activities” course assumed the possibility of presenting the results of learning by the means of social networks, which could have a positive effect on the students’ involvement and progress.

Thus, it is necessary to conduct further research, considering both the listed factors and blended learning in various courses during the entire period of study at the undergraduate level. We suppose a comparative analysis of courses presented on different platforms, but with similar conditions for passing the threshold of success will be very investigating. A possible direction for the study could be the expansion of the sample, the inclusion of students from other Russian and foreign universities. We need further research to compare the peculiarities of the educational process and the learning outcomes of the students from different blended courses, which allow identifying the critical factors that should influence the construction of the course.

5. Conclusions

In this study, using the data from learning portfolios of students the authors compared a number of aspects of students’ activities from the online learning component of courses of different academic orientation. It was found that for theoretical courses, the predominance of e-learning negatively affected the learning process and results of students’ academic progress. As a conclusion, some recommendations were formulated, for instance, how to change the blended learning format for the “History” course with a decrease in the electronic component. Regarding the “Project activity foundations" course, it could be stated that the existing combination of e-learning and face-to-face classes as the form of blended learning was appropriate. However, the time spent by students for online activities in both courses was significantly less than recommended by developers. It may indicate both deficiencies in planning and lack of students’ involvement in this form of education that requires additional research and a redesign of courses.