ASSESSMENT, MEASUREMENT AND EVALUATION CONFERENCE

Previously Published Papers on "Assessment, Measurement and Evaluation Conference"

• Readiness of Intellectual Capital Measurement: A Review of the Property Development and Investment Industry

  Authors: Edward C. W. Chan, Benny C. F. Cheung, Keywords: Intellectual capital, intellectual capital measurement, property development, property investment, Skandia Navigator, VAIC. DOI:10.5281/zenodo. Abstract: In the knowledge economy, the financial indicator is not the unique instrument to gauge the performance of a company. The role of intellectual capital contributing to the company performance is increasing. To measure the company performance due to intellectual capital, the value-added intellectual capital (VAIC) model is adopted to measure the intellectual capital utilization efficiency of the subject companies. The purpose of this study is to review the readiness of measuring intellectual capital for the Hong Kong listed companies in the property development and property investment industry by using VAIC model. This study covers the financial reports from the representative Hong Kong listed property development companies and property investment companies in the period 2014-2019. The findings from this study indicated the industry is ready for IC measurement employing VAIC framework but not yet ready for using the extended VAIC model.

• Impact on Course Registration and SGPA of the Students of BSc in EEE Programme due to Online Teaching during the COVID-19 Pandemic

  Authors: Muhibul Haque Bhuyan, Keywords: electrical and electronic engineering students, impact on course grading and SGPA, online assessment, online teaching, student registration, semester result DOI:10.5281/zenodo. Abstract: Most educational institutions were compelled to switch over to the online mode of teaching, learning, and assessment due to the lockdown when the corona pandemic started around the globe in the early part of the year 2020. However, they faced a unique set of challenges in delivering knowledge and skills to their students as well as formulating a proper assessment policy. This paper investigates whether there is an impact on the student Semester Grade Point Average (SGPA) due to the online mode of teaching and learning assessment at the Department of Electrical and Electronic Engineering (EEE) of Southeast University (SEU). Details of student assessments are discussed. Then students’ grades were analyzed to find out the impact on SGPA based on the z-test by finding the standard deviation (). It also pointed out the challenges associated with the online classes and assessment strategies to be adopted during the online assessment. The student admission, course advising, and registration statistics were also presented in several tables and analyzed based on the change in percentage to observe the impact on it due to the pandemic. In summary, it was observed that the students’ SGPAs are not affected but student course advising and registration were affected slightly by the pandemic. Finally, the paper provides some recommendations to improve the online teaching, learning, assessment, and evaluation system.

• Scholar Index for Research Performance Evaluation Using Multiple Criteria Decision Making Analysis

  Authors: C. Ardil, Keywords: Multiple Criteria Decision Making Analysis, MCDMA, Research Performance Evaluation, Scholar Index, h index, Science Citation Index, Science Efficiency, Cumulative Citation Index, Sciencemetrics DOI:10.5281/zenodo. Abstract: This paper aims to present an objective quantitative methodology on how to evaluate individual’s scholarly research output using multiple criteria decision analysis. A multiple criteria decision making analysis (MCDMA) methodological process is adopted to build a multiple criteria evaluation model. With the introduction of the scholar index, which gives significant information about a researcher's productivity and the scholarly impact of his or her publications in a single number (s is the number of publications with at least s citations); cumulative research citation index; the scholar index is included in the citation databases to cover the multidimensional complexity of scholarly research performance and to undertake objective evaluations with scholar index. The scholar index, one of publication activity indexes, is analyzed by considering it to be the most appropriate sciencemetric indicator which allows to smooth over many drawbacks of scholarly output assessment by mere calculation of the number of publications (quantity) and citations (quality). Hence, this study includes a set of indicators-based scholar index to be used for evaluating scholarly researchers. Google Scholar open science database was used to assess and discuss scholarly productivity and impact of researchers. Based on the experiment of computing the scholar index, and its derivative indexes for a set of researchers on open research database platform, quantitative methods of assessing scholarly research output were successfully considered to rank researchers. The proposed methodology considers the ranking, and the selection of data on which a scholarly research performance evaluation was based, the analysis of the data, and the presentation of the multiple criteria analysis results.

• The Impact of ISO 9001 Certification on Brazilian Firms’ Performance: Insights from Multiple Case Studies

  Authors: Matheus Borges Carneiro, Fabiane Letícia Lizarelli, José Carlos de Toledo, Keywords: Balanced scorecard, Brazilian firms’ performance, critical success factors, ISO 9001 certification, performance measurement. DOI:10.5281/zenodo. Abstract: The evolution of quality management by companies was strongly enabled by, among others, ISO 9001 certification, which is considered a crucial requirement for several customers. Likewise, performance measurement provides useful insights for companies to identify the reflection of their decision-making process on their improvement. One of the most used performance measurement models is the balanced scorecard (BSC), which uses four perspectives to address a firm’s performance: financial, internal process, customer satisfaction, and learning and growth. Since ISO 9001 certified firms are likely to measure their performance through BSC approach, it is important to verify whether the certificate influences the firm performance or not. Therefore, this paper aims to verify the impact of ISO 9001:2015 on Brazilian firms’ performance based on the BSC perspective. Hence, nine certified companies located in the Southeast region of Brazil were studied through a multiple case study approach. Within this study, it was possible to identify the positive impact of ISO 9001 on firms’ overall performance, and four Critical Success Factors (CSFs) were identified as relevant on the linkage among ISO 9001 and firms’ performance: employee involvement, top management, process management, and customer focus. Due to the COVID-19 pandemic, the number of interviews was limited to the quality manager specialist, and the sample was limited since several companies were closed during the period of the study. This study presents an in-depth analysis of how the relationship between ISO 9001 certification and firms’ performance in a developing country is.

• Toward Discovering an Architectural Typology Based on the Theory of Affordance

  Authors: Falntina Ahmad Alata, Natheer Abu Obeid, Keywords: Affordance theory, affordance categories, architectural environments, architectural evaluation criteria, emotional experience. DOI:10.5281/zenodo. Abstract: This paper revolves around the concept of affordance. It aims to discover and develop an architectural typology based on the ecological concept of affordance. In order to achieve this aim, an analytical study is conducted and two sources were taken into account: 1- Gibson's definition of the concept of affordance and 2- The researches that are concerned on the affordance categorisation. As a result, this paper concluded 16 typologies of affordances, including the possibilities of mixing them based on both sources. To clarify these typologies and provide further understanding, a wide range of architectural examples are presented and proposed in the paper. To prove this vocabulary’s capability to diagnose and evaluate the affordance of different environments, an experimental study with two processes have been adapted: 1. Diagnostic process: the interpretation of the environments with regards to its affordance by using the new vocabulary (the developed typologies). 2. Evaluating process: the evaluation of the environments that have been interpreted and classified with regards to their affordances. By using the measures of emotional experience (the positive affect ‘PA’ and the negative affect ‘NA’) and the architectural evaluation criteria (beauty, economy and function). The experimental study proves that the typologies are capable of reading the affordance within different environments. Additionally, it explains how these different typologies reflect different interactions based on the previous processes. The data which are concluded from the evaluation of measures explain how different typologies of affordance that have already reflected different environments had different evaluations. In fact, some of them are recommended while the others are not. In other words, the paper draws a roadmap for designers to diagnose, evaluate and analyse the affordance into different architectural environments. After that, it guides them through adapting the best interaction (affordance category), which they intend to adapt into their proposed designs.

• Research on the Teaching Quality Evaluation of China’s Network Music Education APP

  Authors: Guangzhuang Yu, Chun-Chu Liu, Keywords: Network music education APP, teaching quality evaluation, index, connotation. DOI:10.5281/zenodo. Abstract: With the advent of the Internet era in recent years, social music education has gradually shifted from the original entity education mode to the mode of entity plus network teaching. No matter for school music education, professional music education or social music education, the teaching quality is the most important evaluation index. Regarding the research on teaching quality evaluation, scholars at home and abroad have contributed a lot of research results on the basis of multiple methods and evaluation subjects. However, to our best knowledge the complete evaluation model for the virtual teaching interaction mode of the emerging network music education Application (APP) has not been established. This research firstly found out the basic dimensions that accord with the teaching quality required by the three parties, constructing the quality evaluation index system; and then, on the basis of expounding the connotation of each index, it determined the weight of each index by using method of fuzzy analytic hierarchy process, providing ideas and methods for scientific, objective and comprehensive evaluation of the teaching quality of network education APP.

• Evaluation of Pragmatic Information in an English Textbook: Focus on Requests

  Authors: Israa A. Qari, Keywords: EFL, Requests, Saudi, speech acts, textbook evaluation. DOI:10.5281/zenodo. Abstract: Learning to request in a foreign language is a key ability within pragmatics language teaching. This paper examines how requests are taught in English Unlimited Book 3 (Cambridge University Press), an EFL textbook series employed by King Abdulaziz University in Jeddah, Saudi Arabia to teach advanced foundation year students English. The focus of analysis is the evaluation of the request linguistic strategies present in the textbook, frequency of the use of these strategies, and the contextual information provided on the use of these linguistic forms. The researcher collected all the linguistic forms which consisted of the request speech act and divided them into levels employing the CCSARP request coding manual. Findings demonstrated that simple and commonly employed request strategies are introduced. Looking closely at the exercises throughout the chapters, it was noticeable that the book exclusively employed the most direct form of requesting (the imperative) when giving learners instructions: e.g. listen, write, ask, answer, read, look, complete, choose, talk, think, etc. The book also made use of some other request strategies such as ‘hedged performatives’ and ‘query preparatory’. However, it was also found that many strategies were not dealt with in the book, specifically strategies with combined functions (e.g. possibility, ability). On a sociopragmatic level, a strong focus was found to exist on standard situations in which relations between the requester and requestee are clear. In general, contextual information was communicated implicitly only. The textbook did not seem to differentiate between formal and informal request contexts (register) which might consequently impel students to overgeneralize. The paper closes with some recommendations for textbook and curriculum designers. Findings are also contrasted with previous results from similar body of research on EFL requests.

• Guidelines for Developing, Supervising, Assessing and Evaluating Capstone Design Project of BSc in Electrical and Electronic Engineering Program

  Authors: Muhibul Haque Bhuyan, Keywords: Course outcome, capstone design project, assessment and evaluation, electrical and electronic engineering. DOI:10.5281/zenodo. Abstract: Inclusion of any design project in an undergraduate electrical and electronic engineering curriculum and producing creative ideas in the final year capstone design projects have received numerous comments at the Board of Accreditation for Engineering and Technical Education (BAETE) several times by the mentors and visiting program evaluator team members at different public and private universities in Bangladesh. To eradicate this deficiency which is needed for getting the program accreditation, a thorough change was required in the Department of Electrical and Electronic Engineering (EEE) for its BSc in EEE program at Southeast University, Dhaka, Bangladesh. We suggested making changes in the course curriculum titles and contents, emphasizing to include capstone design projects, question setting, examining students through other standard methods, selecting and retaining Outcome-Based Education (OBE)-oriented engineering faculty members, improving laboratories through purchasing new equipment and software as well as developing new experiments for each laboratory courses, and engaging the students to practical designs in various courses and final year projects. This paper reports on capstone design project course objectives, course outcomes, mapping with the program outcomes, cognitive domain of learning, assessment schemes, guidelines, suggestions and recommendations for supervision processes, assessment strategy, and rubric setting, etc. It is expected that this will substantially improve the capstone design projects offering, supervision, and assessment in the undergraduate EEE program to fulfill the arduous requirements of BAETE accreditation based on OBE.

• Educational Path for Pedagogical Skills: A Football School Experience

  Authors: A. Giani, Keywords: Relational needs, responsibility, self-evaluation, values. DOI:10.5281/zenodo. Abstract: The current pedagogical culture recognizes an educational scope within the sports practices. It is widely accepted, in the pedagogical culture, that thanks to the acquisition and development of motor skills, it is also possible to exercise abilities that concern the way of facing and managing the difficulties of everyday life. Sport is a peculiar educational environment: the children have the opportunity to discover the possibilities of their body, to correlate with their peers, and to learn how to manage the rules and the relationship with authorities, such as coaches. Educational aspects of the sport concern both non-formal and formal educational environments. Coaches play a critical role in an agonistic sphere: exactly like the competencies developed by the children, coaches have to work on their skills to properly set up the educational scene. Facing these new educational tasks - which are not new per se, but new because they are brought back to awareness - a few questions arise: does the coach have adequate preparation? Is the training of the coach in this specific area appropriate? This contribution aims to explore the issue in depth by focusing on the reality of the Football School. Starting from a possible sense of pedagogical inadequacy detected during a series of meetings with several football clubs in Piedmont (Italy), there have been highlighted some important educational needs within the professional training of sports coaches. It is indeed necessary for the coach to know the processes underlying the educational relationship in order to better understand the centrality of the assessment during the educational intervention and to be able to manage the asymmetry in the coach-athlete relationship. In order to provide a response to these pedagogical needs, a formative plan has been designed to allow both an in-depth study of educational issues and a correct self-evaluation of certain pedagogical skills’ control levels, led by the coach. This plan has been based on particular practices, the Educational Practices of Pre-test (EPP), a specific version of community practices designed for the extracurricular activities. The above-mentioned practices realized through the use of texts meant as pre-tests, promoted a reflection within the group of coaches: they set up real and plausible sports experiences - in particular football, triggering a reflection about the relationship’s object, spaces, and methods. The characteristic aspect of pre-tests is that it is impossible to anticipate the reflection as it is necessarily connected to the personal experience and sensitivity, requiring a strong interest and involvement by participants: situations must be considered by the coaches as possible settings in which they could be found on the field.

• Assessing and Evaluating the Course Outcomes of Electrical Circuit Course for Bachelor of Science in Electrical and Electronic Engineering Program

  Authors: Muhibul Haque Bhuyan, Sher Shermin Azmiri Khan, Keywords: OBE, COs, POs, assessment and evaluation, electrical circuit course. DOI:10.5281/zenodo. Abstract: At present, it is an imperative and stimulating task to grow the concepts and skills of undergraduate students in any course. Educators must build up students' higher-order complex and critical thinking abilities. But many of them find it difficult to assess and evaluate these abilities of students who undertake their courses during undergraduate studies. In this research work, a simple assessment and evaluation process for the electrical circuit course of the undergraduate Electrical and Electronic Engineering (EEE) program is reported using the Outcome-Based Education (OBE) approach. The methodology of the work, course contents design, course outcomes (COs) preparation and mapping it with program outcomes (POs), question setting following Bloom's taxonomy, assessment strategy of the students, CO and PO evaluation records, statistics, and charts have been reported for a student-cohort of electrical circuit course taken in Spring 2019 Semester at EEE Department of Southeast University (SEU). It is found that the benchmark fixed by the course instructor has been achieved by the students of that course through CO assessment and evaluation. Recommendations of the course teacher for further quality enhancement based on CO achievement are also presented.