Recommendations for the evaluation of educational stem projects – Department of Software Systems and Technologies

APPROVED
at the meeting of the
Department of software systems and technologies
Minutes No. 16 dated May 01, 2026.

1. Purpose and principles

According to the Concept for the Development of Natural and Mathematical Education, approved by the Cabinet of Ministers of Ukraine on August 5, 2020, stem education forms competencies using a transdisciplinary approach to learning. This approach is based on the practical application of scientific, mathematical, technical and engineering knowledge and skills for their further use in professional activities.
For the formation of competencies provided by the educational programs of the Department of Software Systems and Technologies, the stem project is an important component that implements the principles of stem education within software engineering.
By the number of participants, an educational stem project can be individual or team. According to the number of components within which the project is implemented, the student has the right to choose a single-component or multicomponent stem project. Subject to the joint implementation of the task by representatives of different departments or specialties, the stem project is classified as interdepartmental or intersectoral.

Evaluation of a stem project is carried out according to the following basic principles:

Multicriteria with the use of weighting coefficients. Each criterion has a fixed weight. The result according to a separate criterion is calculated in proportion to the percentage of its implementation.
Objectivity and measurability. The level of performance is determined solely by quantitative metrics. These include: the share of successfully completed test cases, the percentage of code coverage by tests, the absence or presence of comments from static code analysis tools (for example, cpplint or Roslyn Analyzers), compliance with deadlines for tasks and compliance with checklists. This minimizes the subjectivity of the assessment.
Differentiation in the complexity of the individual zone in team projects. At the initial stage of control, each applicant's personal area of responsibility and the level of its complexity are recorded.
Academic integrity as a defining condition. Observance of the principles of academic integrity is applied not as a separate criterion, but as an external factor to the total score.

An educational stem project is considered credited only if the final percentage of its implementation by the applicant is not less than 60 %.

2. Formulas for calculating the assessment for the project

Score for i-th criterion ( $K_i$ ) as a percentage is calculated by the formula:

K_i=k_i \cdot r_i,

Wherein $k_i$ – weight fraction i-th criterion; $r i$ – level of implementation i-th criterion (in percent).
Final percentage of project implementation ( $P$ ) is defined as:

P = M \cdot \sum_{i=1}^n K_i,

where n is the total number of criteria; M is the academic integrity multiplier (see section 3).

Final number of points for the project ( $B$ ) is calculated by the proportion:

B = \frac{P \cdot B_{max}}{100\%},

Wherein $B_{max}$ – the maximum number of points provided for in the draft work program of the discipline (LTP).

3. Academic Integrity and AI Policy

Observance of the principles of academic integrity is applied as an external factor M to the final score:

M = 1.0 – no violations were detected.
M = 0.5 – violations were detected: plagiarism, borrowing code/layouts without proper attribution, undeclared use of generated code.
M = 0.0 – gross falsification of results or impersonation of someone else's work. In this case, the final score is zero, and the materials are submitted to the relevant commission.

Policy on Generative AI and Related Tools and Technologies

The use of metaprogramming tools in a broad sense, in particular AI code generation tools and other services related to the generation and automatic processing of texts in formal and informal languages, during the project work is allowed (and, depending on the task of the work, may be recommended) under the following prerequisites:

Declaration: in the META-USAGE file, the applicant describes the applied programming principles (purpose), their advantages and limitations in the context of project work, and also indicates the specific tools used, key requests for them, generated code snippets, etc.;
Material proficiency: during the defense, the applicant demonstrates knowledge of the content of the META-USAGE file, in particular the ability to answer additional questions about the chosen strategy for using the mentioned tools and reasonably compare it with alternative approaches studied within this or previous disciplines;
Understanding of the generated content: in defense, the applicant confirms understanding of the content of the generated code fragments or artifacts, ways to check their quality (in particular, correctness, effectiveness, etc.) and possible ways of their further use.

The undeclared generated result qualifies as a violation of academic integrity in accordance with “Regulations on Ensuring Academic Integrity at Taras Shevchenko National University of Kyiv", enacted by Rector's Order No. 569-32 dated October 05, 2022.

4. Principles of team stem project evaluation

Project assessment is an assessment of THE APPLICANT for work in a team project, not a team assessment.

To ensure objectivity, the evaluation criteria are divided into two components:

The team unit makes up 30% of the overall project score. The team creates a single product, so its quality is a shared responsibility and is evaluated equally for all team members. This forms the right motivation: the applicant is interested not only in performing his own area of work, but also in bringing the common product to working condition and successful integration of all components.
The individual part is 70% of the overall project evaluation. Individual contribution to the project is assessed for each applicant separately.

Below are two headings: for programming and mathematical disciplines.

4.1. Section A · Programming / Databases

Block C – command component (the same for everyone in the team). Blocks A, B, D, E are individual. Data sources: version control system (Git: issue → branch → PR committees → → review → merge), task tracker, tool reports (coverage, linter), fixed set of acceptance test cases, protection.

Code	Criterion	Weight ( $k$ )	Criterion level ( $r$ ), %
A. Process and discipline (individual)
A1	Compliance with intermediate deadlines (3 checkpoints)	0,06	All 3 checkpoints passed on time with a complete list of → 100% artifacts; 1 stage with a delay ≤24 h → 80%; 2 of 3 on time → 60%; 1 of 3 → 40%; none or post factum → 0%.
A2	Repository /logbook maintenance (Git history check)	0,05	20% for each item: (1) ≥1 substantive commit per week; (2) committees tied to the issue; (3) README with launch instructions; (4) solution log ≥5 “→solution problem” entries; (5) using a branch model (avoiding direct commit in main).
A3	Feedback processing	0,04	The total percentage of criterion fulfillment = the percentage of processed comments from checkpoints issued as an issue and closed until the next stage (0–100%).
B. Contribution and reliability (Individual)
B1	Fulfillment of the individual area of responsibility	0,12	The zone is fixed at the initial stage of the project. The level is assessed by a set of artifacts (issue → branch → PR committees → review → merge chain; code, architectural solutions, tests, documentation, debugging); story points are only one of → the evidence, not an end in itself. Full volume qualitatively → 100 %; mainly (minor gaps) → 80 %; partially → 60 %; fragmentary → 40 %; minimum → 20 %; → 0 % not performed.Authorship is determined by the work performed, not by git blame.
B2	Fulfillment of obligations and risk communication	0,08	The total percentage of criterion fulfillment = the percentage of tasks taken that are EITHER completed within the agreed time frame, OR for which the risk/delay in advance (before the deadline) is communicated with the issue update and the new date (0–100 %). Silent failure of the deadline – not counted (0 %).
B3	Code-review performance	0,05	Only effective reviews are counted (the review of someone else's PR → found a significant problem or suggested a reasonable improvement, the → author responded with a → verified correction). ≥3 effective → 100 %; 2 → 80 %; 1 → 50 %; 0 → 0 %. Small cosmetic comments are not taken into account.
C. Technical quality of the product (TEAM – the same for all)
C1	Functional correctness (acceptance cases)	0,12	The total percentage of criterion fulfillment = passed acceptance test cases / total number of agreed cases (fixed list of cases) × 100 %.
C2	Handling of extreme cases and errors	0,05	20 % for each item: (1) input validation; (2) processing of empty/incorrect data; (3) processing of exceptions without crashing; (4) boundary values; (5) no raw exceptions in the run log file.
C3	Autotests and Coatings	0,06	100% autotests → + coverage ≥80% (tool report) + CI automatically runs tests; 80% → autotests + coverage ≥60%; 60% → autotests + coverage ≥40%; 40% autotests → are available, coverage is <40%; There are no 0% → autotests.
C4	Code quality by static analysis	0,04	Critical/major comments of the linter analyzer per 1000 lines: 0 → 100%; ≤2 → 80%; ≤5 → 60%; ≤10 → 40%; >10 → 0%. Instrumental check (pylint/ESLint/SonarQube).
C-5	Integrity and safety of working with data /DB schema	0,03	20 % for each item: (1) normalization ≥3NF or reasonable denormalization; (2) PK/FK and integrity constraints; (3) search field/JOIN indexes; (4) parameterized queries (injection protection); (5) transactions where required. For projects without DB – equivalent for file data (validation, transactionality, consistency).
D. Complexity of the area of responsibility (individual)
D1	Complexity and completeness of the individual area of responsibility	0,1	The zone is fixed at the initial stage (for individual projects – the entire project). The total percentage of criterion fulfillment = difficulty level × completeness of fulfillment. Difficulty levels: expert (event-driven backend, complex authorization, distributed workers, etc.) → 1.0; advanced → 0.9; basic → 0.8. Completeness: fully → 100 %, partially/simplified → in proportion to the percentage of work performed (30-80 %); trivial contribution (e.g.README + template CRUD) → 20 %; → 0 % failed.
E. Protection and understanding (individual)
E1	Answers to an open-ended set of questions	0,12	The applicant receives K questions on the published set. Each answer: full → 100 %, partial → 50 %, no → 0 %. The total percentage of the criterion fulfillment = the sum of the percentages for the answers / K.
E2	Modification of On-the-Fly Condition	0,05	Two pre-prepared micro-change conditions. Each correctly explained/implemented → 50 %; partially modified or with errors → 25 %; cannot make changes → 0 %. The total percentage of the criterion fulfillment = the sum of the percentages for the responses.
E3)	README Launch Demonstration	0,03	The project is started from scratch by README without author intervention within the defined time limit of → 100 %; with small intervention of → 50 %; → 0 % is not started.
РАЗОМ (sum of weights)		1,00

4.2. Section B · Mathematical disciplines

Block C – command component. Blocks A, B, D, E are individual. Data sources: workbook and intermediate calculations, reference set of tasks/cases for reconciliation of results, reproducible calculation script, protection.

Code	Criterion	Weight ( $k$ )	Criterion level ( $r$ ), %
A. Process and discipline (individual)
A1	Compliance with intermediate deadlines (3 checkpoints)	0,06	All 3 checkpoints passed on time with a complete list of → 100% artifacts; 1 stage with a delay ≤24 h → 80%; 2 of 3 on time → 60%; 1 of 3 → 40%; none or post factum → 0%.
A2	Keeping a work log / execution of intermediate results	0,05	20 % for each item: (1) weekly progress records; (2) each result is dated and contains a method/source reference; (3) intermediate calculations are saved and reproducible; (4) error/correction log ≥5 records; (5) final materials are structured (LaTeX/Jupyter Notebook).
A3	Feedback processing	0,04	The total percentage of criterion fulfillment = the percentage of processed comments from checkpoints taken into account and closed until the next stage (0–100%).
B. Contribution and reliability (individual)
B1	Execution of the individual area of responsibility (deduction / calculation / sections)	0,12	The zone is fixed at the initial stage of the project. The level is assessed by the population: inferences performed, computational experiments, sections, checks (confirmed by the workbook/repository). Full volume qualitatively → 100 %; mainly (minor gaps) → 80 %; partially → 60 %; fragmentary → 40 %; minimum → 20 %; → 0 % not performed.
B2	Fulfillment of obligations and risk communication	0,08	The total percentage of criterion fulfillment = the percentage of tasks taken that are EITHER completed within the agreed time frame, OR for which the risk/delay in advance (before the deadline) is communicated with the update of the record and the new date (0-100%). Silent failure of the deadline – not counted (0 %).
B3	Effectiveness of mutual verification of solutions	0,05	Only effective reviews are credited (checked the solutionof a colleague → found an error or suggested a significant improvement, the → author responded with a → verified correction). ≥3 effective → 100%; 2 → 80%; 1 → 50%; → 0 0%.
C. Mathematical quality (TEAM – the same for all)
C1	Correctness of results (reference set)	0,12	The total percentage of criterion fulfillment = tasks/cases with a result that coincides with the reference/analytical solution within the given error / total number of cases × 100 %.
C2	Rigor of justification	0,06	20 % for each item: 1) all inference steps are given; 2) named theorems/methods with conditions; 3) conditions of applicability of the method are checked; 4) error/convergence assessment is given; 5) verification of dimensions/limiting cases.
C3	Adequacy of selected mathematical methods and models	0,05	25 % for each item: 1) assumptions are explicitly written out; 2) verification on a known individual case; 3) analysis of sensitivity to parameters; 4) comparison with an alternative method/source.
C4	Numerical correctness and stability	0,04	25 % for each item: 1) processing of extreme cases (division by zero, poor conditioning); 2) control of rounding error; 3) numerically confirmed convergence; 4) reproducibility (fixed data/seed).
C-5	Notation and reproducibility of design	0,03	33 % for each item: 1) correct and consistent mathematical notation; 2) all designations are defined; 3) reproducible script/calculation layouts are provided.
D. Complexity of the area of responsibility (individual)
D1	Complexity and completeness of the individual area of responsibility	0,1	The zone is fixed at the initial stage (for individual projects – the entire project). The total percentage of criterion fulfillment = difficulty level × completeness of fulfillment. Difficulty levels: expert (non-standard proof, research/optimization task) → 1.0; advanced → 0.9; basic → 0.8.Completeness: fully → 100%, partially/simplified → depending on the percentage of work performed (30-80%); trivial contribution → 20%; not completed → 0%.
E. Protection and understanding (individual)
E1	Answers to an open bank of theoretical questions	0,12	The applicant receives K questions on the published set. Each answer: full → 100 %, partial → 50 %, no → 0 %. The total percentage of the criterion fulfillment = the sum of the percentages for the answers / K.
E2	Solvinga modified problem "on the fly"	0,05	Two pre-prepared microchanges of parameters/conditions. Each solved/explained → 50%; partially solved or with errors → 25 %; not solved → 0 %. The total percentage of the criterion fulfillment = the sum of the percentages for the responses.
E3)	Demonstration of calculation reproducibility	0,03	Recalculation: from scratch reproduces the result → 100%; partially → 50%; no → 0%.
РАЗОМ (sum of weights)		1,00

4.3. Mechanics of data collection

To ensure theobjectivity ofthe assessment, confirmation of the results by the actual development artifacts and systematic monitoring of progress during the semester, four main stages are used:

Initial stage of the project. The applicant fixes the work plan and determines the personal area of responsibility in the team, indicating its level of complexity (basic, advanced or expert). For example: development of API and authorization module; configuration of persistence level and migrations; implementation of asynchronous handlers and caching.The developed terms of reference serve as a basis for further assessment of the quality of the process and the complexity of the individual contribution.
Intermediate control points (Checkpoints). During the semester, three control points are provided (for example, at the 2nd, 5th and 8th weeks). The current results of the development and the quality of the organization of the process are subject to assessment. All detected comments are recorded as issues and are subject to mandatory elimination until the next stage, which makes it possible to measure the dynamics of correcting deficiencies.
Open Question Bank: A list of questions on the relevant topics is published in advance. Project protection is carried out individually for each applicant and covers both the processing of his individual area of responsibility and an understanding of the overall architecture of the product. Completeness of responses is assessed against a clear checklist (complete, partial, or missing), making protection the primary verification tool for real-world contributions.
Anonymous peer-review as an indicator of anomalies. The results of mutual evaluation by team members do not automatically change the final score. In case of significant discrepancies between the assessments of team members, the teacher conducts an additional individual check.

4.4. Example: one team – different semester grades

The team made a good product, so the team component is the same for everyone: C = 27% (out of 30%).
Applicant A (actively working):
A = 14%, B = 23%, D = 9%, E = 18% → in total 64%.
The final percentage of project implementation by the applicant is 27% + 64% = 91%.
Applicant B (worked less):
A = 12%, B = 14%, D = 7%, E = 16% for a → total of 49%.
The final percentage of project implementation by the applicant is 27% + 49% = 76%.
Applicant V (almost “passenger”):
A = 5%, B = 4%, D = 3%, E = 8% for a → total of 20%.
The final percentage of project implementation by the applicant is 27% + 20% = 47%.
This is BELOW the 60% threshold, so the draft is not drawn up.

As a result: one product, one team – but different ratings. The project is not saved by either a joint product or laboratory, if personal participation in the project is below the threshold.

5. Principles of evaluation of individual and other stem projects

The principles of evaluation are adapted depending on the format and scale of the project, but the basic criterion is always the measurability of the results in accordance with the approved Terms of Reference (ToR).

Individual stem projects: Evaluation is carried out according to a table of criteria developed for team projects, provided that the team consists of one participant. In this case, the applicant assumes full responsibility for the entire development life cycle. Accordingly, the individual and "team" (product) components are combined, and 100% of the final score is formed solely on the basis of the personal result of the applicant.
Multicomponent stem projects. If the project covers several academic disciplines (educational components), the assessment is carried out in a differentiated manner for each of them. Each discipline assesses separately, based on the degree of achievement of those competencies and the fulfillment of those functional requirements that were clearly assigned to this component in the joint terms of reference.This allows you to enroll one large-scale project as a result of studying several subjects at once without "double" scoring for the same work.
Interdepartmental and intersectoral stem projects. Projects with a high level of integration are subject to separate assessment by experts of each involved department or industry. Points are awarded independently, based on the fulfillment of the specific requirements of each subject area prescribed in the ToR.Protection of such projects can take place in the format of joint intersectoral commissions or through successive defenses in the relevant departments, which provides an objective professional assessment of the applicant's cross-functional skills.

Annex A. Recommendations for the assessment of laboratory and homework

Laboratory and homework is a mandatory INDIVIDUAL component of the semester assessment. Their purpose is to check the completeness of the acquisition of competencies by each applicant. The work is assessed not on the principle of "handed over / not handed over", but in aggregate:

75% – correctness of execution
25% – short individual protection (2–3 minutes of oral questions).

Protection inherits AI policy: if the applicant cannot explain its own result, the "correctness" part is not counted. Protection can be carried out selectively (some applicants per pair).
Each work has a "raw" weight in complexity, and its performance is estimated as a percentage. The result is normalized, so the number of works and their complexity can be any:

L = L_{max} × \frac{\sum_{i=1}^n w_i \cdot s_i}{\sum_{i=1}^n w_i \cdot 100\%}

Wherein $L$ – final assessment for all laboratory and homework; $L,MAX$ – the maximum possible final assessment; $W + I$ – work complexity weight coefficient (simple ≈ 5, average ≈ 8, complex ≈ 12); $% S (*% I)$ – evaluation of work as a percentage (0–100%); n – total number of works (up to 12); i – number of current work, $i \in \overset{\rule{1.5em}{0.4pt}}{{1,n}}$ .