1. Introduction

COVID-19 disease crisis has had a strong impact on higher education system, specifically on medical education system, which includes a lot of practical teaching and working with patients (Karimian et al., 2021). To prevent the spread of the infection, measures of physical distance were introduced, and distance learning had to be held. Restricting physical presence has accelerated the development of an online learning environment (Papapanou et al., 2021). Although there were tools for online teaching before the pandemic, it was necessary to improve them to withstand a huge number of users and enable uninterrupted teaching. In addition to those technical problems, the need for educating teachers and students in the field of digital technologies has been identified so they can adapt to sudden change in the learning and teaching system (Rashid & Yadav, 2020). A qualitative study by Ahmed et al. (2000) showed that distance learning is a good environment for most students, although it can be challenging for students of lower socio-economic status and students with disabilities. Face-to-face learning achieves better results when it comes to teacher-student connection, acquiring competence of professionalism through teamwork, cognitive, communication and clinical skills.

At the School of Medicine in Zagreb, MEF-LMS platform (Copyright © School of Medicine in Zagreb) (Žižak, 2009), which is continuously being improved, is primarily used in teaching. Zoom application (Copyright ©2021 Zoom Video Communications, Inc., USA) (Zoom, 2021) is also used to relieve sometimes-overloaded MEF-LMS system. Over time, teaching that includes theoretical knowledge has been adapted for online teaching, while practical parts of teaching have been reduced and condensed in an epidemiologically appropriate manner, or delayed due to an increase in cases of SARS-CoV-2 virus infection. The situation was further worsened by earthquakes that devastated the buildings of the School of Medicine as well as a significant part of the clinical bases where students performed clinical practice.

Lack of practical training was compensated by development of digital materials, but they proved to be not as good as acquiring practical skills live, especially when it comes to working with patients (Langenau et al., 2017). Currently, despite the unfavourable epidemiological situation, face-to-face learning is allowed throughout the country, under epidemiological measures. Due to extensive reconstruction of damaged buildings of the School of Medicine which are necessary for normal functioning of the faculty, some classes will have to be held online in the future. Regarding the professional qualification of medical doctors, it is important to ensure quality of education, face-to-face as well as distance learning (Monier et al., 2019). Evaluation of educational process is also important part of enabling the quality of education.

 

1.1. Aim and hypothesis

Aim: To assess medical students’ perception of quality of educational process during the COVID-19 pandemic.

Hypothesis: There will be a difference in the perception of quality of educational process during the COVID-19 pandemic among fourth-, fifth-, and sixth-year medical students.

2. Methods

2.1. Participants

This research included 142 medical students (43 male and 99 female), from fourth, fifth and sixth year of School of Medicine, University of Zagreb, Croatia (Table 1).

2.2. Materials

Croatian version of newly developed and validated Questionnaire for Assessing Quality of Distance Learning in Biomedicine Studies (Cronbach alpha = 0.86) was used as measurement instrument. We presented challenges in development of the questionnaire and the results of the questionnaire validation at EDULEARN21 conference, and we described them in the paper published in the EDULEARN21 Proceedings (Nola et al., 2021). The complete questionnaire is presented for the first time in this paper. Validated Croatian version of this questionnaire is available upon request.

The questionnaire consists of seven open-ended questions and 28 items, rated on Likert scale from 1 to 7 (1 means 'I completely disagree'; 2 – 'I mostly disagree'; 3 – 'I partially disagree'; 4 – 'I neither agree nor disagree; 5 – I partially agree; 6 – I mostly agree; 7 – I completely agree) and distributed in six subscales (Table 3 – Table 9): Perception of Distance Learning Organization (N = 6), Perception of Educational Environment and Personal Academic Achievements (N = 5), Perception of Personal Academic Activity (N = 4), Perception of Cooperation (N = 6), Perception of Equipment Quality (N = 3) and Perception of Teachers’ Work (N = 4)  (Nola et al., 2021). The content of these subscales is shown in Tables 4, 5, 6, 7, 8 and 9.

2.3. Procedure

This research was conducted during May 2021 among students from School of Medicine, University of Zagreb. The questionnaire was created in Google Forms and link was given to students during regular online classes and then shared through social networks. Students did not need to be authenticated by AAI@EduHr system to access the questionnaire. Participation in this questionnaire was anonymous and voluntary, and students could exit the questionnaire at any time. The Ethical Committee of School of Medicine, University of Zagreb approved development of this questionnaire as well as this research (Record number: 380-59-10106-21-111/106; Class: 641-01/21-02/105).

2.4. Statistical methods

Each result of descriptive analysis was shown as percentage, i.e., as mean value with standard deviation (SD). Shapiro-Wilk test was used to assess normality of data distribution. One-Way ANOVA with Bonferroni post-hoc test was used to determine statistically significant differences between mean values of items and subscales of the questionnaire regarding the year of study of the respondents. All p values lower than 0.05 were considered statistically significant. The results expressed as mean values were interpreted according to the range which belongs to every individual point of Likert scale (1 – ‘Very bad’: 1.00 - 1.85; 2 – ‘Rather bad’: 1.86 - 2.71; 3 – ‘Bad’: 2.72 - 3.57; 4 – ‘Neither good nor bad’: 3.58 - 4.43; 5 – ‘Good’: 4.44 - 5.29; 6 – ‘Rather good’: 5.30 - 6.15; 7 – ‘Very good’: 6.16 - 7.00) (Pimentel, 2019). All statistical analyses were conducted using IBM^® SPSS^® Statistics for Windows (Version 25.0).

 

3. Results

Out of 142 respondents, 34 of them (23.9 %) were fourth-, 19 (13.4 %) were fifth- and 89 (62.7 %) were sixth-year medical students (Table 1). More than half of all respondents (N = 90, 63.4 %) had never before the pandemic participated in any kind of distance learning. According to the Croatian National Grading Scale, where excellent (5) is the highest grade, the overall Grade Point Average (GPA) of all participants was 4.16 ± 0.44 (Table 1). The most common grade during distance learning was excellent (74.6 %), while during face-to-face learning it was very good (47.2 %) (Table 1, Table 2).

Table 1. Answers to open-ended questions

Open-ended questions	Answers
Study program:	Integrated Study of Medicine in Croatian
Year of study:	4th	5th	6th
Gender (N(%)):	M: 9 (6.34)   F: 25 (17.61)	M: 4 (2.82)    F: 15 (10.56)	M: 30 (21.13)   F:  60 (42.25)
GPA rounded to two decimal places (Mean ± SD):	4.02 ± 0.58	3.93 ± 0.45	4.26 ± 0.34
The most common grade during distance learning:	Excellent (5)	Excellent (5)	Excellent (5)
The most common grade during face-to-face learning:	Excellent (5)	Very good (4)	Very good (4)
Have you ever participated in any form  of distance learning before? N(%):	Yes: 7 (20.59)    No: 27 (79.41)	Yes: 2 (10.53)   No: 17 (89.47)	Yes: 44 (48.89)  No: 46 (51.11)

There was no statistically significant difference in overall GPA regarding the year of study. However, there was statistically significant difference (p < 0.001) when it comes to grades good (3), very good (4) and excellent (5), obtained during distance learning and face-to-face learning, among sixth-year students. Sixth-year students were more successful in exams during distance learning compared to face-to-face learning (7.5% less grades good, 38.9 % less grades very good and 46.7 % more grades excellent) (Table 2).

Table 2. Distribution of the most common grade per year of study and their statistical difference in relation to the type of teaching

Year of study	Type of teaching	Grade (N(%))				F	p	Bonferroni's < 0.05
		Sufficient (2)	Good (3)	Very good (4)	Excellent (5)	df = 2
4	DL	1 (2.94)	4 (11.76)	13 (38.24)	16 (47.06)	2.726	0.062
	FTF	2 (5.88)	9 (26.47)	11 (32.35)	12 (35.29)
5	DL	0 (0.00)	0 (0.00)	6 (31.58)	13 (68.42)	0.480	0.701
	FTF	1 (5.26)	5 (26.32)	11 (57.89)	2 (10.53)
6	DL	0 (0.00)	2 (22.00)	11 (12.22)	77 (85.56)	15.505	< 0.001	Grade good, very good and excellent
	FTF	0 (0.00)	9 (10.00)	46 (51.11)	35 (38.89)

DL – distance learning, FTF – face-to-face learning

The overall Questionnaire score (Table 3) was 4.42 ± 0.72 out of maximum 7 (63.1 % of the total score), and it showed statistically significant difference (p = 0.012) among fourth- and sixth-year students. The fourth-year students had significantly lower score (4.11 ± 0.81) compared to that of the sixth-year students (4.54 ± 0.65). The mean score and interpretation of results of each subscale is shown in Table 1. All subscales mean scores were above 50.0 % of the total score, except the subscales Perception of Distance Learning Organization (47.4 % of the total score) and Perception of Personal Academic Activity (47.9 % of the total score).

The lowest mean value for subscale Perception of Distance Learning Organization (Table 4) was found in fourth year (3.01 ± 1.04) and the highest in sixth year (3.47 ± 1.04). Here, the most problematic Items were Item 2 and Item 4. Item 2 was mostly rated in range from 1 to 3 on Likert scale. The fifth-year students predominantly chose 1 (N = 6, 31.6 %), sixth-year students mostly chose 2 (N = 24, 27.0 %), while fourth-year students mostly chose 3 (N = 10, 26.5 %) on Likert scale. Item 4 was predominantly rated with 1 on Likert scale (N = 65, 45.8 % of all participants). 64.7 % (N = 22) of fourth-year students and 36.8 % (N = 7) of fifth-year students rated this Item with 1 on Likert scale. Statistically significant difference for Item 6 was found between fourth and sixth year (p < 0.001) and between fifth and sixth year (p = 0.020) (Table 4).

In the subscale Perception of Educational Environment and Personal Academic Achievements (Table 5) statistically significant difference was found in Item 7, Item 9, Item 10, and Item 11 between fourth and fifth (p < 0.001, p = 0.014, p < 0.001 and p = 0.008, respectively), and fourth and sixth year (p < 0.001, p < 0.001, p < 0.001 and p = 0.005, respectively). All items were overall rated more positively than negatively (Mean: ≥ 4.43), but fourth-year students predominantly chose 1 on Likert scale for Item 7 (N = 9, 26.5 %), Item 9 (N = 9, 26.5 %) and Item 10 (N = 10, 29.4 %) (Table 5).

The mean score for subscale Perception of Personal Academic Activity (Table 6) was significantly higher (p < 0.001) for fourth year (3.51 ± 1.32), compared to sixth year (3.29 ± 1.08). All items were rated more negatively than positively (Mean: 2.71 – 3.72). There were no statistically significant differences in Items regarding the years of study (Table 6).

In the subscale Perception of Cooperation (Table 7) statistically significant difference (p = 0.039) was found in Item 21 between fifth and sixth year. All items are mostly positively rated (Mean: ≥ 4.08) except Item 20 (Mean: 3.18). The fourth-year students rated this Item more negatively than students of other years (Mean: 2.79) (Table 7).

The highest mean score was found for subscale Perception of Equipment Quality (87.4 % of the total score) (Table 8). All Items in this subscale were rated predominantly in range from 6 to 7 on Likert scale. There were no statistically significant differences in Items regarding the year of study (Table 8).

In the subscale Perception of Teachers’ Work (Table 9) statistically significant difference (p < 0.001, p = 0.012) was found in Item 26 and Item 28 between fourth and sixth year. Item 26 and Item 28 were the worst rated from fourth-year students (23.5 %, N = 8, chose 1; 26.5 %, N = 9, chose 2 on Likert scale) and the best from sixth-year students (24.7 %, N = 22, chose 6 on Likert scale; 25.8 %, N = 23, chose 6 on Likert scale). All items were mostly positively rated (Mean: 4.32 – 5.43) except Item 27 (Mean: 3.83) (Table 9).

Table 3. Interpretation of the results of the questionnaire and its subscales by year of study and statistical differences

Subscale	Year of study	N	Mean	SD	Subscale interpretation	F	p	Bonferroni's < 0.05
						df = 2
Perception of Distance Learning Organization	4	34	3.01	1.04	Bad	2.46	0.089
	5	19	3.18	1.25	Bad
	6	89	3.47	1.04	Bad
	All	142	3.32	1.08	Bad
Perception of Educational Environment and Personal Academic Achievements	4	34	3.84	1.67	Neither good nor bad	18.51	< 0.001	4th and 5th 4th and 6th
	5	19	5.42	1.14	Rather good
	6	89	5.30	1.07	Rather good
	All	142	4.97	1.39	Good
Perception of Personal Academic Activity	4	34	3.51	1.32	Bad	0.41	0.664
	5	19	3.32	1.52	Bad
	6	89	3.29	1.08	Bad
	All	142	3.35	1.20	Bad
Perception of Cooperation	4	4	4.02	1.11	Neither good nor bad	1.81	0.168
	5	19	4.07	1.02	Neither good nor bad
	6	89	4.39	1.04	Neither good nor bad
	All	142	4.26	1.06	Neither good nor bad
Perception of Equipment Quality	4	34	6.22	1.04	Very good	0.52	0.598
	5	19	6.28	1.01	Very good
	6	89	6.04	1.17	Rather good
	All	142	6.12	1.12	Rather good
Perception of Teachers' Work	4	34	4.06	1.35	Neither good nor bad	4.55	0.012	4th and 6th
	5	19	4.42	1.30	Neither good nor bad
	6	89	4.73	0.96	Good
	All	142	4.53	1.14	Good
Questionnaire	4	34	4.11	0.81	Neither good nor bad	5.60	0.005	4th and 6th
	5	19	4.45	0.60	Good
	6	89	4.54	0.65	Good
	All	142	4.42	0.72	Neither good nor bad

Table 4. Distribution of percentage and mean value of answers on items in the subscale
Perception of  Distance Learning Organization N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
1. Materials utilized in e-learning systems are of good quality.	4th	2.9	14.7	14.7	26.5	20.6	17.6	2.9	4.12 ± 1.49
	5th	5.3	21.1	26.3	15.8	10.5	5.3	15.8	3.84 ± 1.86
	6th	3.4	5.6	15.7	27.0	24.7	18.0	5.6	4.40 ± 1.42
	All	3.5	9.9	16.9	25.4	21.8	16.2	6.3	4.26 ± 1.51
2. Distance learning is focused on acquiring the most important knowledge and skills that students should have at the end of their education.	4th	17.6	26.5	29.4	8.8	2.9	11.8	2.9	3.00 ± 1.67
	5th	31.6	10.5	15.8	15.8	15.8	10.5	0.0	3.05 ± 1.81
	6th	11.2	27.0	21.3	21.3	10.1	5.6	3.4	3.22 ± 1.54
	All	15.5	24.6	22.5	17.6	9.2	7.7	2.8	3.15 ± 1.60
3. Online classes, seminars and practicals are interactive.	4th	5.9	8.8	17.6	23.5	20.6	20.6	2.9	4.18 ± 1.55
	5th	15.8	21.1	15.8	5.3	42.1	0.0	0.0	3.37 ± 1.61
	6th	5.6	15.7	22.5	24.7	22.5	7.9	1.1	3.71 ± 1.39
	All	7.0	14.8	20.4	21.8	24.6	9.9	1.4	3.77 ± 1.47
4. In person practicals organized as a part of distance learning, allow students to get all the necessary practical skills.	4th	64.7	11.8	8.8	5.9	8.8	0.0	0.0	1.82 ± 1.34
	5th	36.8	31.6	0.0	5.3	21.1	5.3	0.0	2.58 ± 1.77
	6th	40.4	27.0	9.0	10.1	10.1	1.1	2.2	2.35 ± 1.57
	All	45.8	23.9	7.7	8.5	11.3	1.4	1.4	2.25 ± 1.56
5. The e-learning system allows classes to be of high quality.	4th	32.4	14.7	20.6	14.7	11.8	2.9	2.9	2.79 ± 1.68
	5th	10.5	15.8	15.8	36.8	0.0	10.5	10.5	3.74 ± 1.79
	6th	14.6	14.6	18.0	19.1	20.2	10.1	3.4	3.60 ± 1.70
	All	18.3	14.8	18.3	20.4	15.5	8.5	4.2	3.42 ± 1.73
6. Students are encouraged to take part in teamwork.	4th	38.2	32.4	11.8	14.7	0.0	2.9	0.0	2.15 ± 1.26
	5th	31.6	26.3	15.8	21.1	0.0	5.3	0.0	2.47 ± 1.43
	6th	12.4	19.1	13.5	29.2	12.4	11.2	2.2	3.53 ± 1.61
	All	21.1	23.2	13.4	24.6	7.7	8.5	1.4	3.06 ± 1.62

Table 5. Distribution of percentage and mean value of answers on items in the subscale
Perception of Educational Environment and Personal Academic Achievements, N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
7. I have more free time.	4th	26.5	14.7	14.7	2.9	5.9	14.7	20.6	3.74 ± 2.39
	5th	0.0	5.3	5.3	5.3	10.5	31.6	42.1	5.84 ± 1.46
	6th	2.2	6.7	2.2	9.0	11.2	22.5	46.1	5.72 ± 1.65
	All	7.7	8.5	5.6	7.0	9.9	21.8	39.4	5.26 ± 2.01
8. My environment during distance learning allows me to fulfil my obligations with high quality.	4th	11.8	11.8	8.8	11.8	11.8	14.7	29.4	4.62 ± 2.17
	5th	5.3	0.0	0.0	15.8	26.3	21.1	31.6	5.47 ± 1.54
	6th	1.1	5.6	7.9	14.6	13.5	22.5	34.8	5.40 ± 1.62
	All	4.2	6.3	7.0	14.1	14.8	20.4	33.1	5.23 ± 1.78
9. I like the flexibility of online classes.	4th	26.5	5.9	17.6	17.6	5.9	8.8	17.6	3.68 ± 2.20
	5th	5.3	5.3	0.0	10.5	31.6	26.3	21.1	5.21 ± 1.62
	6th	5.6	4.5	4.5	10.1	12.4	28.1	34.8	5.43 ± 1.76
	All	10.6	4.9	7.0	12.0	13.4	23.2	28.9	4.98 ± 1.99
10. It is easier for me to pass my exams.	4th	29.4	17.6	11.8	17.6	11.8	2.9	8.8	3.09 ± 1.94
	5th	5.3	10.5	5.3	15.8	10.5	21.1	31.6	5.05 ± 1.96
	6th	2.2	7.9	5.6	29.2	18.0	20.2	16.9	4.81 ± 1.57
	All	9.2	10.6	7.0	24.6	15.5	16.2	16.9	4.43 ± 1.87
11. I fulfil my academic obligations quickly and efficiently.	4th	8.8	23.5	11.8	11.8	17.6	5.9	20.6	4.06 ± 2.06
	5th	0.0	5.3	0.0	15.8	21.1	31.6	26.3	5.53 ± 1.35
	6th	2.2	6.7	6.7	12.4	23.6	25.8	22.5	5.16 ± 1.59
	All	3.5	10.6	7.0	12.7	21.8	21.8	22.5	4.94 ± 1.75

Table 6. Distribution of percentage and mean value of answers on items in the subscale
Perception of Personal Academic Activity, N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
12. I often use my camera and microphone.	4th	14.7	47.1	11.8	2.9	5.9	11.8	5.9	2.97 ± 1.85
	5th	31.6	42.1	5.3	5.3	0.0	10.5	5.3	2.53 ± 1.87
	6th	0.0	18.0	40.4	13.5	16.9	9.0	2.2	2.65 ± 1.32
	All	7.7	28.2	28.9	9.9	12.0	9.9	3.5	2.71 ± 1.54
13. I join online classes. but I do not participate. R	4th	8.8	23.5	17.6	17.6	8.8	23.5	0.0	3.65 ± 1.70
	5th	15.8	21.1	26.3	5.3	10.5	10.5	10.5	3.47 ± 1.98
	6th	4.5	23.6	27.0	19.1	11.2	12.4	2.2	3.55 ± 1.50
	All	7.0	23.2	24.6	16.9	10.6	14.8	2.8	3.56 ± 1.61
14. I do other things during online classes. R	4th	17.6	11.8	23.5	14.7	8.8	14.7	8.8	3.65 ± 1.94
	5th	10.5	15.8	31.6	21.1	10.5	5.3	5.3	3.42 ± 1.58
	6th	7.9	25.8	27.0	19.1	11.2	7.9	1.1	3.28 ± 1.43
	All	10.6	21.1	26.8	18.3	10.6	9.2	3.5	3.39 ± 1.58
15. I actively participate in online classes.	4th	5.9	32.4	17.6	5.9	14.7	8.8	14.7	3.76 ± 1.97
	5th	5.3	31.6	15.8	10.5	10.5	10.5	15.8	3.84 ± 2.01
	6th	7.9	20.2	16.9	20.2	22.5	11.2	1.1	3.67 ± 1.54
	All	7.0	24.6	16.9	15.5	19.0	10.6	6.3	3.72 ± 1.71

*R – reverse coded

Table 7. Distribution of percentage and mean value of answers on items in the subscale
Perception of Cooperation, N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
16. It is easy for students to collaborate with teachers during distance learning.	4th	2.9	23.5	20.6	20.6	5.9	14.7	11.8	3.94 ± 1.79
	5th	15.8	10.5	10.5	36.8	10.5	15.8	0.0	3.63 ± 1.64
	6th	2.2	14.6	16.9	21.3	22.5	14.6	7.9	4.22 ± 1.57
	All	4.2	16.2	16.9	23.2	16.9	14.8	7.7	4.08 ± 1.63
17. I can get all the necessary information from the administrative staff.	4th	8.8	2.9	20.6	32.4	14.7	14.7	5.9	4.09 ± 1.56
	5th	15.8	5.3	21.1	10.5	26.3	10.5	10.5	4.00 ± 1.92
	6th	6.7	6.7	11.2	15.7	21.3	29.2	9.0	4.52 ± 1.73
	All	8.5	5.6	14.8	19.0	20.4	23.2	8.5	4.35 ± 1.72
18. I actively collaborate with my colleagues.	4th	8.8	17.6	11.8	14.7	26.5	8.8	11.8	4.06 ± 1.84
	5th	0.0	5.3	31.6	26.3	21.1	5.3	10.5	4.21 ± 1.40
	6th	4.5	4.5	20.2	19.1	25.8	18.0	7.9	4.43 ± 1.52
	All	4.9	7.7	19.7	19.0	25.4	14.1	9.2	4.31 ± 1.59
19. I easily get information from my colleagues regarding news about the subjects I attend.	4th	5.9	0.0	2.9	23.5	26.5	11.8	29.4	5.18 ± 1.62
	5th	0.0	5.3	5.3	15.8	26.3	31.6	15.8	5.21 ± 1.36
	6th	2.2	5.6	2.2	10.1	27.0	30.3	22.5	5.35 ± 1.47
	All	2.8	4.2	2.8	14.1	26.8	26.1	23.2	5.29 ± 1.49
20. If necessary, teachers are willing to adapt content to students with special needs (e.g., dyslexia).	4th	29.4	8.8	20.6	35.3	5.9	0.0	0.0	2.79 ± 1.37
	5th	10.5	15.8	10.5	31.6	10.5	15.8	5.3	3.84 ± 1.74
	6th	15.7	21.3	11.2	42.7	2.2	2.2	4.5	3.19 ± 1.51
	All	18.3	17.6	13.4	39.4	4.2	3.5	3.5	3.18 ± 1.53
21. I can get all the necessary documents from the administrative staff.	4th	11.8	8.8	5.9	35.3	17.6	11.8	8.8	4.09 ± 1.73
	5th	10.5	31.6	10.5	10.5	21.1	10.5	5.3	3.53 ± 1.84
	6th	6.7	10.1	9.0	18.0	21.3	24.7	10.1	4.62 ± 1.68
	All	8.5	12.7	8.5	21.1	20.4	19.7	9.2	4.35 ± 1.75

Table 8. Distribution of percentage and mean value of answers on items in the subscale
Perception of Equipment Quality, N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
22. The equipment that I use during distance learning is of good quality.	4th	0.0	2.9	8.8	0.0	2.9	20.6	64.7	6.24 ± 1.39
	5th	0.0	0.0	10.5	5.3	0.0	26.3	57.9	6.16 ± 1.34
	6th	2.2	2.2	3.4	5.6	13.5	25.8	47.2	5.92 ± 1.44
	All	1.4	2.1	5.6	4.2	9.2	24.6	52.8	6.03 ± 1.41
23. I own my own computer with Internet connection, which I use to do everything necessary for distance learning.	4th	0.0	0.0	2.9	0.0	2.9	2.9	91.2	6.79 ± 0.77
	5th	0.0	0.0	0.0	0.0	10.5	5.3	84.2	6.74 ± 0.65
	6th	1.1	0.0	1.1	1.1	0.0	6.7	89.9	6.79 ± 0.85
	All	0.7	0.0	1.4	0.7	2.1	5.6	89.4	6.78 ± 0.80
24. The Internet network I use allows me to fulfil my obligations without any problems.	4th	2.9	0.0	8.8	8.8	20.6	17.6	41.2	5.62 ± 1.56
	5th	5.3	5.3	0.0	0.0	5.3	36.8	47.4	5.95 ± 1.68
	6th	5.6	4.5	12.4	4.5	9.0	20.2	43.8	5.43 ± 1.91
	All	4.9	3.5	9.9	4.9	11.3	21.8	43.7	5.54 ± 1.80

Table 9. Distribution of percentage and mean value of answers on items in the subscale
Perception of Teachers’ Work, N(4^th) = 34, N(5^th) = 19, N(6^th) = 89

Item		Likert scale							Mean ± SD
	Year	1	2	3	4	5	6	7
		% of answers per year
25. Teachers follow instructions set by the Faculty regarding written and oral exams.	4th	11.8	2.9	8.8	11.8	14.7	17.6	32.4	4.97 ± 2.05
	5th	5.3	5.3	0.0	21.1	15.8	21.1	31.6	5.26 ± 1.76
	6th	1.1	0.0	7.9	13.5	13.5	30.3	33.7	5.64 ± 1.38
	All	4.2	1.4	7.0	14.1	14.1	26.1	33.1	5.43 ± 1.63
26. Teachers have demonstrated an understanding for students who have difficulties in following online classes.	4th	23.5	8.8	11.8	20.6	11.8	17.6	5.9	3.65 ± 1.98
	5th	0.0	10.5	26.3	21.1	15.8	10.5	15.8	4.37 ± 1.64
	6th	4.5	4.5	13.5	13.5	21.3	24.7	18.0	4.89 ± 1.67
	All	8.5	6.3	14.8	16.2	18.3	21.1	14.8	4.52 ± 1.81
27. Teachers are skilled in using all the necessary tools for distance learning.	4th	5.9	20.6	17.6	14.7	14.7	23.5	2.9	3.94 ± 1.72
	5th	0.0	21.1	21.1	21.1	21.1	10.5	5.3	3.95 ± 1.51
	6th	6.7	15.7	28.1	11.2	21.3	15.7	1.1	3.76 ± 1.56
	All	5.6	17.6	24.6	13.4	19.7	16.9	2.1	3.83 ± 1.58
28. Students are informed well in advance about their assignments.	4th	8.8	26.5	17.6	11.8	17.6	5.9	11.8	3.68 ± 1.87
	5th	5.3	26.3	10.5	10.5	21.1	10.5	15.8	4.11 ± 1.97
	6th	1.1	7.9	11.2	24.7	23.6	25.8	5.6	4.62 ± 1.39
	All	3.5	14.8	12.7	19.7	21.8	19.0	8.5	4.32 ± 1.64

4. Discussion

At the School of Medicine in Zagreb, most of the clinical classes are held in the fourth and fifth year, while the sixth year encompasses less clinical classes and more public health subjects that can be more easily adapted to distance learning (Crvenković et al., 2018).

Most medical students expect a lot of practical education, and the highest expectations are from fourth-year students who have the most clinical subjects. During this pandemic period, students have not had sufficient opportunity to attend practical or clinical classes. Our research showed that, compared to face-to-face learning, students were more successful in exams during distance learning (1.4 % less grades sufficient, 12.0 % less grades good, 26.8 % less grades very good, 40.1 % more grades excellent). This could be related to insufficient monitoring of students during the exams. However, since the School of Medicine organized supervised distance exams, we are not convinced that the improvement in exam performance can be attributed exclusively to the use of materials which are not allowed during face-to-face exams (Hassan et al., 2020). We allow the possibility that students were more relaxed (less stressed) or that the teachers might have lowered their criteria due to circumstances of increased stress during the COVID-19 pandemic and earthquakes, which influenced this change. Students’ ethical and moral integrity and values which could make them prone to cheating during online exams were not examined by this questionnaire. However, a research review of academic integrity in online assessment by Holden et al. (2021) showed that both, students and faculty perceive online testing to offer more cheating opportunities than in traditional, live-proctored classroom environment.

Assessment of the quality of educational process showed that the highest quality of distance learning was perceived by sixth-year (64.9 % of the total score), and the lowest by fourth-year medical students (58.7 % of the total score). This result might be influenced by the fact that the fourth-year students did not have enough face-to-face clinical classes, unlike the sixth-year students who had attended almost the entire course live, in full capacity, before the pandemic. This is in accordance with the results of research conducted at this faculty by Crvenković et al. (2018), which assessed the educational environment during face-to-face learning. At that time, the lowest quality of the educational environment was perceived by sixth-year students, who had the least clinical practice.

In general, students considered distance learning neither good nor bad, although they perceived it more positively than negatively (4.42 ± 0.72). Medical students were not satisfied with organization of classes (N = 89, 62.7 %), primarily because distance learning did not enable the acquisition of knowledge and skills that students should have at the end of their study. This was confirmed by 75.99 % respondents in the research conducted by Dost et al. (2021). Our students had good prerequisites for participation in distance learning i.e. a computer with Internet connection, which they could use to perform all their obligations related to distance learning (N = 138, 97.2 %) and adequate Internet connection (N = 109, 76.8 %). This is in accordance with research conducted by Puljak et al. (2020) (86.0 % and 83.7 %, respectively). Although MEF-LMS system had been used at the School of Medicine for many years, there were some technical difficulties in full transition to online classes. The system was overloaded with many users causing occasional breakdowns during classes, especially if camera and/or microphone were used. This is one possible reason why students indicated that classes was often not interactive (N = 60, 42.3 %), and they only passively participated in it (N = 78, 54.9 %). Furthermore, even with the improvement of the system, students did not want to actively participate in the classes, or they did other things during classes (N = 83, 58.5 %). Teachers mainly used MEF-LMS system for their real-time presentations and notes. For other activities in MEF-LMS, according to 47.9 % (N = 68) of all participating students, teachers were not sufficiently educated nor skilled enough in using the necessary tools for online classes. Over time, educational workshops were held to train the teachers to use the system more operatively.  

5. Conclusion

During the pandemic, distance learning has become an integral part of the higher education system, even in those areas where it was unthinkable until recently. Results of our study showed that students’ perception of teachers’ work, educational environment and personal academic achievement vary among fourth, fifth and sixth year of study. This perception is mainly related to how much of practical education they get, and how qualified the teachers are to use the online tools during distance learning. Majority of students were satisfied with equipment quality and Internet connection they used, but most of them thought that teachers were not qualified enough to use the tools for online teaching. Distance learning may become increasingly incorporated into existing curricula in the future. It is still necessary to educate teachers to use digital technology in the best possible way in order to achieve educational outcomes. Students in biomedical field, who, due to the pandemic, could not attend practicals or had a reduced number of them, should have extra practical classes. However, organization of future teaching processes should anticipate this problem and adapt all educational elements to distance learning. Furthermore, it is necessary to conduct regular evaluation of practical classes, especially in times of crisis. It is important to highlight that, although the students had mainly positive perception of organization of distance learning, they still did not actively participate in classes. This part of distance learning needs additional attention.

6. References

Ahdika, A. (2017). Improvement of Quality, Interest, Critical, and Analytical Thinking Ability of Students through the Application of Research Based Learning (RBL) in Introduction to Stochastic Processes Subject. International Electronic Journal of Mathematics Education, 12(2), 167-191.

Ahmed, S. A. et al. (2020). Model for utilizing distance learning post COVID-19 using (PACT)™ a cross sectional qualitative study. BMC Medical Education, 20(1), 400.

Crvenkovic, M. et al. (2018). Medical students' perception of educational environment: 6 years integrated medical school. In: ICERI Proceedings. Seville (Spain): IATED; 6302-310.

Dost, S. et al. (2020). Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open, 10(11): e042378.

Hassan, B. et al. (2020). Online assessment for the final year medical students during covid-19 pandemics; the exam quality and students' performance. Onkologia i Radioterapia, 14.

Holden, O. L., Norris, M. E., & Kuhlmeier, V. A. (2021). Academic Integrity in Online Assessment: A Research Review. Frontiers in Education, 6(258).

Karimian, Z. (2021) et al. Medical education and COVID-19 pandemic: a crisis management model towards an evolutionary pathway. Education and Information Technologies. 2021.

Langenau, E., Lee, R. & Fults, M. (2017). Blended Learning Educational Format for Third-Year Pediatrics Clinical Rotation. The Journal of American Osteopathic Association, 17, 234-43.

Monier, E. B. et al. (2019). Student Evaluation of Distance Learning for Health Care Professionals. Telemedicine Journal and e-Health, 25(6), 485-91.

Nola, I. A. et al. (2021). Challenges in the development of questionnaire for assessing quality of distance learning in biomedicine studies. In: EDULEARN21 Proceedings. Valencia (Spain): IATED; 7696-703.

Papapanou, M. et al. (2021). Medical education challenges and innovations during COVID-19 pandemic. Postgraduate Medical Journal, 0, 1-7.

Pimentel, J. L. (2019). Some Biases in Likert Scaling Usage and its Correction. International Journal of Sciences: Basic and Applied Research, 45, 183-91.

Puljak, L. et al. (2020). Attitudes and concerns of undergraduate university health sciences students in Croatia regarding complete switch to e-learning during COVID-19 pandemic: a survey. BMC Medical Education, 20(1), 416.

Rashid, S. & Yadav, S. S. (2020). Impact of COVID-19 Pandemic on Higher Education and Research. Indian Journal of Human Development, 14, 340-43.

Zoom. (2021). Retrieved December 20, 2021, from https://zoom.us/

Žižak, M. (2019). Autorsko pravo i pravo korištenja autorskih djela u LMS okruženju. LMS Medicinskog fakulteta. Sveučilište u Zagrebu. Retrieved December 28, 2021, from Mef.hr website: https://lms.mef.hr/main/attachments/article/5/Zizak_M_Autorska_prava_u_LMS_okruzenju.pdf