GO4FUTURESKILLS – A COMPREHENSIVE COMPETENCY ASSESSMENT TOOL

: The aim of the article is to analyse the possibility of using the elements of a multidimensional comparative analysis in order to create a tool enabling a comprehensive assessment of a person’s competencies, and comparing it with the requirements set in a specific vocational situation. An illustration of such possibilities is the presentation of a tool for assessing competencies prepared under the international Go4FutureSkills project. A competency profile was defined as a set of desirable competencies with the level of knowledge or skills required for a given profession/position assigned to each of them. The competency matrix created in this way serves as a basis for the comparison of the results obtained by the people using the tool. In its present form, this tool can be used by students and graduates majoring and specialising in logistics, as well as by employees from the logistics industry.


Introduction
The rapid increase in the amount of information resulting from the enormous number of available sources has prompted changes in teaching methods and a shift from teaching to learning. G. Siemens, one of the founders of connectivism, or the theories of learning in the digital age, advises teachers not to teach content but rather the methods of obtaining it. In connectivism, more important than answering the question of what you know at a particular moment is the question of whether you know where, and how to find the necessary knowledge (Siemens, 2005;Duke, Harper, and Johnston, 2013). This is because knowledge is important to us when we want to apply it, and if we do not know how to update it, it may appear to be useless. On the other hand, the competencies and needs of the labour market desired by employers are changing, while technological progress makes these changes happen increasingly faster. Knowledge, which until recently seemed to be extremely modern, soon turns out to be out of date, and the skills acquired during formal education are insufficient. In this context, the possibility of a comprehensive assessment of the competencies of a student, a graduate, a young or an experienced employee in relation to the requirements of a given position or profession is extremely important for the effectiveness of activities in the field of formal education, or within the framework of the idea of lifelong learning. As for higher education, it is helpful to conduct cooperation with the practitioners and the environment of the university. This issue was addressed through the triple helix model created by Leydesdorff, 1995, 2000) which assumes the close cooperation and intermingling of roles between science, business and public administration.
The aim of the article is to analyse the possibility of using the elements of a multidimensional comparative analysis in order to create a tool enabling the comprehensive assessment of a person's competencies, and comparing it with the requirements set in a specific situation (Panek and Zwierzchowski, 2013;Walesiak and Gatnar, 2013;Walesiak, 2011). An illustration of such possibilities will be the presentation of a tool for assessing competencies and formulating development proposals prepared under the international Go4FutureSkills project 1 . In its present form this tool can be used by students and graduates majoring and specialising in logistics, as well as by employees from the logistics industry.
The main idea of the Go4FutureSkills Model was to create a modern mechanism, in which competency profiles of professions tailored to the needs and expectations and tools for measuring the level of competencies used in the described profiles are presented, and moreover, proposals for development activities aimed at improving the competencies desirable on the market are described. It is worth emphasising that although the Go4FutureSkills Model solutions in the project were developed as a tool supporting the employees of the logistics industry, it is of a much wider and flexible nature. As a rule, the logistics industry is only an example of the functioning 1 The Go4FutureSkills project (No. POWR.04.03.00-00-0031/18) is the outcome of international cooperation aimed at developing effective solutions adjusting the education and training system to the needs of the labour market. It is financed by the European Social Fund and implemented as part of the Polish-Finnish partnership. On the part of Poland, the institution leading the project is the training company Dobre Kadry Research-Training Centre Sp. Z o. o., which has been operating on the Lower Silesian market for 14 years. On the side of Finland, the entity participating in the project is the Taitaja Adult Education Centre in Kouvola, highly valued by employers from the Kymenlaakso region and active on the market for over 40 years. of the developed tool in specific conditions. The solution was constructed in the form of 'drawers' that can be replaced, adapting to the needs of a specific industry. These replaceable drawers are primarily the so-called knowledge competencies, namely the areas of specialist knowledge necessary to work in a given industry. Replacing them with topics relevant for another industry/field of study can easily facilitate the use of this tool in industries not related to logistics or the supply chain.

Description of the components of the Go4FutureSkills Model and stages of work
The Go4FutureSkills Model consists of four components: Component 1. Competency profiles for professions in the logistics industrya description of the competencies required in the industry, including information on their desirable level for selected professions/positions. Both the hard skills related to specialist knowledge, and the soft ones of a more universal nature were taken into consideration.
Component 2. A tool for assessing the level of the selected competencies -a set of questions, tasks and issues to be solved by the person whose level of competencies is being evaluated. Substantially developed by an interdisciplinary team of experts, and prepared in the form of a program that can be used on a computer and on any mobile device (a phone or tablet).
Component 3. Competency matrix -the rules and tools enabling the comparison of the level of competencies of a given person with the profiles of selected professions/positions in the logistics industry, and an assessment of how similar they are. Prepared in the form of a program that can be used on a computer or any mobile device, and integrated with the software from Component 2.
Component 4. A program offering development activities for students and employees of the logistics industry -a set of proposals aimed at increasing knowledge and improving the competencies important on the labour market.
The mechanism of forecasting skills and competencies necessary to effectively perform work in the professions sought-after both currently and in the future, was used. The authors conducted extensive primary research and carried out the analysis of the results which allowed to unambiguously identify the needs of the logistics industry. For this purpose, a literature query was carried out as part of publications indexed in the Web of Science and Scopus databases (i.e. Christopher, 2011;Hopkins and Hawking, 2018;Marr, 2018;McKinnon, Flöthmann, Hoberg, and Busch, 2017;Perboli, Musso, and Rosano, 2018;Schwab, 2016). On its basis, the appropriate tools for conducting primary research in the project were developed. The main aim of all the studies was to define a determinant of effective education in the field of logistics, according to the current and forecasted requirements of employers and the labour market. The primary research conducted from January to November 2019 covered different target groups, intended to be cross-sectional and comparative, and it was assumed that it would integrate data as well as quantitative and qualitative methods (Denzin and Lincoln, 2014). The quantitative research using the auditorium questionnaire method (PAPI) covered a group of students in their final years of studies in the field of logistics and related courses (N = 964). The study involving practitioners began with qualitative research. The authors conducted 19 individual in-depth interviews (IDI) among employers and managers of logistics companies as well as head-hunters from this sector. Two focus group interviews (FGI) were also carried out with secondary school teachers and academic teachers in the field of logistics. The use of the Delphi method (Cieślak, 2001;Cisek, 2009;Matejun, 2012;Metoda…, 2019;Skulimowski, 2018) in a nationwide survey of experts related to the logistics industry (N = 201) provides a particularly valuable source of information.
The study based on desk research analyses also took into account the issue of the new role of universities in the 21st century (Etzkowitz and Leydesdorff, 2000).
Based on the results of the literature query and primary research, a team of experts selected a set of 13 main types of competencies that allow to describe the preparation for the effective performance of a profession/work in a given position. The types of competencies included typical 'knowledge' ones related to the areas of professional knowledge necessary in the logistics industry, hard IT, analytical or language skills, as well as soft and interpersonal skills connected with creativity and learning. Each of the above-mentioned types of competencies was described with the help of several areas of knowledge/scopes of skills that constitute a coherent conglomerate of the required knowledge, skills and attitudes. In the next step, a proposal was developed to define a four-level standard of competencies (understood as knowledge, skills and attitudes in accordance with the National and European Qualifications Framework). Thus a list of 25 professions/positions important from the point of view of the current and future situation in logistics and the supply chain was created, and then ultimately limited to 12 recognised by experts as key for the industry. Competency profiles were created for individual professions/positions by assigning them the required levels for each type of competency. Each profession/position was described by means of 13 types of competencies which were defined at one of the four levels described. The standard of the desirable competencies functions as a stimulant, i.e. the higher the level of a given competency assigned to a given profession, the better the competencies expected in this area for this profession/position. An interdisciplinary team of experts has developed a tool for assessing individual types of competencies in the form of tasks to be solved on one's own. The tool has the form of a multiple-choice test with four possible answers in each question. The person taking part in the assessment receives a randomly selected set of several dozen questions in which they can find four questions from a given type of competency.
For each correct answer, the person undergoing the assessment receives a certain number of points. The total of the points obtained from questions concerning a given type of competency determines the level of competency in this area. Using advanced statistical algorithms, a Competency Matrix was created to compare the competencies of the assessed person with the set of competencies assigned to a given profession/ position in the logistics industry. Within a given type of competency the matrix works like a contingency table, where individual cells contain the result of the assessment of a given competency against its desirable level resulting from the competency profile of the profession. While designing the matrix, the authors used some elements of a multidimensional comparative analysis, in particular including the methods of linear ordering. The assessment of the level of competencies for people doing the test is performed with the use of dedicated software. The program, developed specially for the project, first measures the user's competencies and then compares the results they achieved with the desirable assessments indicated by experts in the profession competency profiles. The assessments are assigned to each profession separately. The system shows where the users obtained the same assessments as those described in the profiles, and where they did not. It also indicates which professions the user taking part in the test is best suitable for. The program makes it possible to export data to MS Excel and generate simple graphs comparing the obtained results (in specific competences) with those desirable in a given profession.
In order to assess the matching of a person's competencies to the requirements set for a specific profession/position, the study used the Euclidean distance. Due to the properties of the distance as the so-called anti-measures, the results considered most compliant with the requirements should be those for which the calculated distances are the smallest.

Competency profiles for selected professions/positions
A competency profile was defined as a set of desirable competencies with the level of knowledge or skills required for a given profession/position assigned to each of them. First the authors selected competencies which in the literature on the subject and according to the respondents, were defined as crucial for the effective functioning of an employee in the logistics industry and the effective performance of their professional duties. The authors distinguished 13 main types: 1. Professional competencies. 2. Analytical skills related to problem solving. 3. Analytical skills related to searching for information, processing and assessment of the usefulness of this information, including work with big data.
4. IT skills related to the knowledge and ability to use a computer as well as basic software.
5. IT skills related to programming and searching through databases. 6. Interpersonal skills. 7. Interpersonal skills when working in a diverse team (different cultures, generations, people with disabilities), displaying appropriate attitudes in this respect.
8. Language skills -the knowledge of foreign languages and its level. 9. Skills in the field of organisation and self-organisation.
10. Skills in the field of creativity (ability to generate new ideas, a creative style of work).
11. Learning skills -openness to constant development and easy adaptation to changes.
12. Personal skills related to loyalty, commitment, responsibility, attitudes towards work.
13. Other skills and competencies, including mainly a holistic approach to issues related to logistics, but also sensitivity to key issues for this industry.
Due to the fact that it is difficult to evaluate a holistic approach without observing a person in real conditions in the workplace, it was decided not to include competency 13 in the Model, whereas the discussed issues were included in other types of competencies. Some of the distinguished competencies were of a complex nature (e.g. professional competencies or soft skills) and were divided into subcategories. Finally, 23 categories of competencies were specified for which four levels were distinguished, described as the standard of the required knowledge, skills and attitudes (in accordance with the Polish and European Qualifications Framework). The description of individual standards for successive levels of competencies is progressive, which means that the next level includes the elements presented in the previous one (see Table 1).

Table 1. Levels of individual types of competencies
Level 1 -Competencies necessary to perform simple and routine tasks which are the basis for further and more specialised activities.
Level 2 -Competencies necessary to perform tasks in typical situations.
Level 3 -Competencies necessary to perform complex tasks, both in typical and problematic conditions.
Level 4 -Competencies necessary to perform many complex tasks of various nature. Leadership and managerial skills. * * Leadership and managerial skills are included only in the selected types of competencies due to the fact that they do not apply to all of them, e.g. professional competencies.
For the 12 professions/positions (see Table 2) which are of key importance for the logistics industry, the authors created competency profiles. The profession/ position profiles were created by assigning the required levels of individual types of competencies to the above-mentioned professions/positions. Table 3 presents the levels of specific competencies indicated by a group of experts in the field of logistics for the selected 12 professions/positions of high importance in the sector of logistics. The competency matrix created in this way serves as a basis for the comparison of the results obtained by the people using the tool. It is worth emphasizing that the competence matrix is highly universal. Replacing the tests checking 'knowledge' from the field of logistics with those checking expertise in other fields of study/ sectors, and developing levels of competency for new selected professions as recommended by experts will make it possible to use in other areas.   4  3  3  3  3  3  3  3  2  3  3  2   12.2 Attitude towards work  4  3  3  3  3  3  2  3  2  2  4  2 12.3 Self-presentation  4  3  3  3  3  3  2  2  2  2  3  2 Source: own elaboration based on expert opinions.

Using the elements of a multidimensional comparative analysis to create a tool for comprehensive competency assessment
As part of the competency matrix, the level of the specified competencies is verified and compared with the pattern (competency profile for the profession). Due to their diverse nature, four variants of asking questions/formulating tasks were prepared.
1. Multiple-choice test questions. These contain four answers, out of which only one is correct. By giving the correct answer one can obtain a certain number of points, but when choosing a different one, no points are scored. This type of questions was mainly used to assess hard skills.
The scheme of the question: Content of the question/task:  first answer  second answer  third answer  fourth answer The outcome variable for this type of question is the following: where x means the number of points assigned to a given task for the correct answer.
2. Questions in which the person undergoing the assessment determines their level of specific competency by selecting one of the four possible answers available that define different levels of a given competency, from the lowest to the highest. Questions of this type were used for self-assessment in the case of competencies for which there are specific standards for each level (knowledge, skills and attitudes were defined, which allow to assign an appropriate level). Language skills (i.e. knowledge of a foreign language) are an example of usage where the Common European Framework of Reference for Languages functions 2 .
The scheme of the question: Self-assessment of a specific competency (e.g. knowledge of the English language):  level 1 (Beginner, A1)  level 2 (Pre-Intermediate, A2)  level 3 (Intermediate, B1)  level 4 (Upper-Intermediate, B2) where determining a given level is a form of self-assessment, e.g. on the basis of a certificate confirming specific knowledge and skills.
3. Questions formulated as descriptions of specific problem situations, where the possible answers reflect possible reactions to specific circumstances. The role of the assessed person is to analyse all variants and determine which of the methods they would choose to solve the problem. For these types of questions there are no correct or incorrect answers. However, it is necessary to note that each possible answer is assigned to a specific level of a given competency. This means that certain types of behaviour indicate a better/poorer awareness of the person of the desirable way of acting in a given problem situation.
The scheme of the question: Description of the problem situation. What would you do? How would you behave in this situation? What activities would you pot for? How would you solve this problem? It should be emphasised that the given ways of acting and the levels specified in the second column of the table appear in a different order, not necessarily from the lowest to the highest level.
The outcome variable for this type of question is the following: where determining a given level of competency is the result of selecting a specific way of acting / behaviour in a given problem situation from among the proposed options.

24
Dorota Kwiatkowska-Ciotucha, Urszula Załuska 4. Questions in which the person undergoing the assessment solves the task and enters a numerical answer. In this situation, there are no variants -only one numerical result is correct. Giving the result requires simple calculations or an analysis of the situation, and a logical answer. A certain number of points is assigned for a correct answer, and if the answer is wrong the person does not obtain any points. Such questions are mainly used to assess analytical skills.
The scheme of the question: Task content. A question. An answer:  space to enter the numerical result The outcome variable for this type of question has a form which is analogous to the variable from the first type of question.
As part of the assessment of the level of a specific competency, one type of question was most often used. Two types were used only when assessing analytical skills within logical thinking and drawing conclusions (the questions described in points 1 and 4).

Calculating the values of partial variables
The authors adopted the following method of calculating the values of partial variables for specific types of questions:

Multiple-choice test questions
The scheme of aggregation of the outcome variables to the partial variables is presented in Figure 1.
Partial variable: the value is determined on the basis of the number of points obtained after adding up the points of the outcome variables, i.e. the points obtained for the correct answers to the questions from 1 to n Go4FutureSkills -a comprehensive competency assessment tool

25
In the case of the multiple-choice questions, the number of points obtained for each randomly selected question within a given competency is added up. The total of the points determines the assignment of this competency to the appropriate level expressed by natural numbers from 0 to 4. On the basis of the obtained number of points, the qualification to the level takes place in the following way: It was assumed that each question has the same points value, and the assignment to the appropriate level of competencies is linear, which means an equal span of value ranges for individual competency levels. At the same time, it was expected that the maximum number of points possible to obtain is divisible by 4 (e.g. 4, 8, 12, 16, etc.), thus the upper limit of the range for the highest level of competencies is also a multiple of 4. For example, if 1 point is obtained for each correct answer and if the number of questions for a given competency is 4, the score of 1 classifies the competency to the first level, the score of 2 -to the second level, the score of 3 -to the third level and the score of 4 -to the fourth and highest level.
2. Questions in which the user determines the level of competency on their own by choosing one of the variants from the list of available options.
The scheme of the transition from the outcome variable to the partial variable is presented in Figure 2. In questions in which the user assesses the level of competency on their own, the values of the partial variable are determined as integers from 1 to 4 according to the level selected by the user, i.e. 1 means the lowest level, whereas 4 -the highest. For example, if the A1 level is indicated when assessing language skills (knowledge of a foreign language), the value of the partial variable will be 1, and the value of the B2 level will be 4.
3. Questions formulated as a description of a specific problem situation.
In such questions, the person assessed within one competency is assigned an appropriate level of this competency based on each problem situation solved (in each question). Then the levels are converted to integers from 1 to 4 according to the number of points obtained by the person in a given question, i.e. 1 is the lowest level, whereas 4 -the highest. In the next step, the values of the partial variable are calculated as the arithmetic mean of these numbers.
The scheme of aggregation of the outcome variables to the partial variables is presented in Figure 3.
Partial variable: arithmetic mean of numbers determined on the basis of the assessment of the level of a given competency in a single question / task  The values of the partial variable are calculated as an arithmetic mean. When the mean is not an integer, they are rounded to 0 decimal places according to the mathematical rounding rules. For example, a person assessed within a given competency on the basis of four questions obtained the following values of the outcome variable: level 2, level 3, level 2 and level 4. These values were converted Go4FutureSkills -a comprehensive competency assessment tool 27 into numbers corresponding to a given level of competency, i.e. 2, 3, 2 and 4. In this situation, the value of the partial variable is 3: ℎ = 2 + 3 + 2 + 4 4 = 2.75 ≈ 3 . 4. Questions in which the user solves a problem and enters a numerical result as an answer.
In the case of questions in which the user has to enter the result obtained on the basis of solving a task, the procedure is analogous to that for the first type of questions. The points obtained for solving specific tasks are counted, and then, on the basis of their total amount, an appropriate level of assessed competency is assigned.
The total of the points obtained determines the assignment of a given competency to the appropriate level, expressed with natural numbers from 0 to 4.
Multiple choice questions and questions which require entering numerical answers can appear together when assessing a single competency. In such a situation, the points from both types of questions are added up, and then the appropriate level of the assessed competency is assigned in accordance with the method provided for the first type of questions. For example, the person undergoing the assessment answered two multiple choice questions and solved two tasks in which they gave a numerical result. For each correct answer, one could obtain 1 point, which means a maximum of 4 points in total (assuming that all the answers are correct). The participants answered two questions correctly and solved one problem correctly, which means that the number of correct answers was 3 and the number of points obtained was also 3.

Calculation of the value of a composite indicator
A composite indicator can be determined in two ways, assuming that the partial variables are stimulants, and that they are stimulants with a veto threshold. In the first situation, for each of the partial variables, the difference between its value and the value defined in the competency profile of a given profession is determined. Next, based on these differences, the distance between the analysed entity and the pattern object (competency profile) is calculated. The basic distance used is the Euclidean one: where: d iw -the distance between the i analysed entity (an assessed person) and the pattern object (a competency profile for a given profession), p -the number of partial variables (the number of assessed types of competencies),

28
Dorota Kwiatkowska-Ciotucha, Urszula Załuska z wk -the level required for the k partial variable in the competency profile (a competency), z ik -the level obtained by the analysed entity for the k partial variable (an assessed competency).
Distances are the measures of dissimilarity between the analysed entity and the pattern object. A greater distance indicates a smaller similarity of the competency level of the assessed person to the one defined in the competency profile of a given profession. The greater the distance, the more the competency level of the assessed person differs from the level of competency required for this profession. It should be noted that when adopting the concept in which the variables are treated as stimulants, the differences between the values of the partial variables for the analysed entity and the pattern object may be both positive and negative. The person undergoing the assessment may have a level of a given competency which is lower or higher than assumed in the competence profile for a given profession. Hence the obtained larger distances do not always indicate the existence of a big competency gap. It should also be noted that their increased values may be the result of a situation in which some competency levels of the assessed person are higher than specified in the profile. Therefore when assessing skills, one should not only rely on distances but also analyse the directions of deviations for specific competencies in a detailed manner.
According to the second concept, the variables are stimulants with a veto threshold. For specific competencies this threshold is the level adopted in the competency profile. In such a situation, the differences between the analysed entity and the pattern object are determined as follows: where the notations are the same as in the Euclidean distance.
This means that if the assessed person has a higher level of a given competency than that assumed in the professional competency profile, it does not affect the distance between these two objects. In this case, greater distances will indicate a larger competency gap. However, this concept does not allow for the discovery of the strengths of the person whose level of competency is assessed, which may lead to erroneous conclusions about the indicated profession. A short distance from the competency profile of a given profession may also occur when the level of most competencies of the assessed person is higher than required (in an extreme case, with a distance equal to zero, all the competencies may be higher than specified in the competency profile). Therefore the authors decided to implement the first concept in the model as the basic one, with the analysis of the directions of deviations from the competency profile for specific competencies and professions.
For each person whose competencies are being assessed, a ranking is created based on the values of distance measures. However, the method of linear ordering in the multidimensional space of competencies is not applied here to assess individual analysed entities and their position in comparison to the others. The basis for creating the ranking is the distance of the analysed entity from the specific pattern objects described by profession competency profiles. The shorter the distance becomes, the closer the level of competencies of the assessed person is to the levels of specific competencies indicated in the competency profile for a given profession. The first place in the ranking is taken by the profession for which the lowest value of the distance measure was noted, while the subsequent ones are ordered according to the values of this measure in an ascending manner (i.e. from the smallest to the largest). The ranking created in this way allows for assessing to what extent the competencies already possessed ensure the possibility of effective employment in specific logistics professions, and to what extent it is necessary to undertake development activities, and in which field (in terms of which competences). In the computer programme developed for assessing competencies and creating a competency matrix, the result of the linear ordering of professions for a given person is presented in Table 4. People whose level of competencies is being assessed also have the possibility of receiving a graphical analysis of the differences within their specific skills. The prepared charts show the direction of discrepancies for specific professions (higher/lower level of a given competency than in the competency profile for a given profession). This visual presentation was prepared in a computer programme for each of the professions/specialisations as shown in Figure 4.   Go4FutureSkills -a comprehensive competency assessment tool

31
The prepared research tool was tested in two groups of participants -final year students of logistics and employees of companies and logistics departments. The aim of the testing was to evaluate the effectiveness of the proposed solution in the context of the verification of learning outcomes in the case of students, and periodic performance reviews in the case of employees. As a general rule, the obtained results could also be a starting point for planning development paths (assessment of the competency gap in light of the performed profession, planned career path change, planned career path after graduation). The assessment of the effectiveness of the proposed solution indicated their extensive application possibilities. On the one hand, the developed computer programme confirmed the adequacy of the results obtained with the assessments of coaches and persons responsible for managing employee teams. On the other hand, it allowed for planning development paths in line with the expectations of employees and employers.

Conclusion
The article presents the tool developed as part of the international Go4FutureSkills project, based on the elements of a multidimensional comparative analysis, which serves to perform a comprehensive assessment of a given person's competencies, and compare this assessment to the requirements set for a specific profession/ position. The first results of testing the tool in the group of students and employees of the logistics industry are promising. However, in the process of analysing its effectiveness, it was possible to observe some problematic issues mainly related to the rules of using the tool by people undergoing the assessment, among others, in the cases of using online resources by respondents, the lack of individual approach, and focusing on 'positive' assessment results instead of concentrating on the benefits of identifying a gap in individual knowledge. What is more, the authors noted that the respondents chose answers (especially in tasks connected with problem situations) which they believed were appropriate to choose, not those which reflected their potential ways of acting. What was positive, however, was the fact that such situations occurred less frequently in the case of professionally active people who already had some professional experience. This increases the chances of using the developed solutions in companies, but requires reflection in the case of students.
The prepared solution has been developed for the logistics industry. As mentioned in the introduction, adapting the tool to the needs of other sectors requires minor modifications, which are mainly connected with the development of questions/ tasks relating to the scope of professional knowledge, specific to a given field of study or industry. Another issue is the assessment of the desirable level of universal competencies that have been developed, defined and assigned to specific levels of knowledge and skills. For other professions/positions, this level will have to be determined on the basis of studies/research in the group of employers, managers and supervisors, namely people who have an impact on the employment of personnel in a given industry.