-
Presentation
Presentation
This course focuses on "data engineering" and its intersection with "data science". In this context, it is intended that students gain technical skills in several independent but related topics. The most relevant areas of this course are databases and programming, which are the fundamental skills one needs to be able to play the role of "data engineer" in academic and/or industrial projects. The inclusion of the course in the masters program is justified with the importance of data collection, validation and processing skills in order to be able to have data that can be "explored" with the knowledge acquired in the other curricular units.
-
Class from course
Class from course
-
Degree | Semesters | ECTS
Degree | Semesters | ECTS
Master Degree | Semestral | 7
-
Year | Nature | Language
Year | Nature | Language
1 | Mandatory | Português
-
Code
Code
ULHT6347-25231
-
Prerequisites and corequisites
Prerequisites and corequisites
Not applicable
-
Professional Internship
Professional Internship
Não
-
Syllabus
Syllabus
Introduction to Data Engineering Git & GitHub Introduction to version control systems Learning elementary work processes using the Git software and the GitHub online platform Databases & SQL Relational Databases SQL language SQL Injection (elementary notions) Python Programming From the data extraction and data processing points of view From the exploratory data analysis point of view Algoritmic complexity and efficiency It's importance when dealing with large amounts of data Jupiter notebook Linux Introduction to the GNU/Linux operating system File system navigation (commands)
-
Objectives
Objectives
Students are expected to learning technical skills related with: - Version control (Git & GitHub) - Relational Data Bases (e.g. MySQL) and SQL - Programming with the Python language, focused on the interaction with relational databases - Elementary notions of algorithmic complexity and efficiency - Elementary notions of the Linux operating system from an end user's perspective It is also expected that the students improve their creativity and critical thinking skills.
-
Teaching methodologies
Teaching methodologies
Theoretical-practical classes with exposition of theory and presentation of practical examples. Exercises to be carried out during the class, with the support and validation of the Teacher. Exercises to do at home. Assessment: 3 mini-tests and a project
-
References
References
Damas, Luís - SQL - Structured Query Language. 14ª edição. Portugal. FCA, 2017. ISBN: 9789727228294
-
Assessment
Assessment
Descrição
Data limite
Ponderação
Teste de avaliação #1
01-Abril-2024
25%
Teste de avaliação #2
01-Junho-2024
25%
Teste de avaliação #3
Projecto
Última aula
Entrega no fim da última semana de aulas.
25%
25%
Espera-se que uma parte significativa do "Projecto" seja desenvolvida durante as aulas. Serão avaliados aspectos técnicos e comportamentais (p.e. interacção com o Professor).
Existirá uma "defesa" do "Projecto", durante a qual serão feitas perguntas e pedidos de alteração do projecto ao aluno, com o objectivo de validar a participação no mesmo.
-
Mobility
Mobility
No
