what is window function in spark-sql ?
Senior Data Engineer Interview Questions
2,552 senior data engineer interview questions shared by candidates
Average questions mostly on SQL, 1 question on data structure. In depth explanation of your current and recent projects. Another interview was based on architecture solutions for certain scenarios.
I got a take home assignment to build SPARK application on recipes dataset. The problem was easy but the expectations on the solution was very high. The code should be object-oriented, with config files, unit test cases, logging and alerting, exception handling. Extra points for CI/CD , etc
Data modelling techniques, how you manage communication with business users, etc
SQL, pyspark, azure data engineering
Mostly ask about hadoop and spark and scala
Spark internal functioning, Data formats
Technical questions regarding the different technology
The Python technical challenge was about ingesting some data in a specific way and do some analytics on the results. The questions were about the reasons behind each decision. Also, they asked for alternatives to the simple provided solution (it was a challenge done in 3h) more oriented to a production-ready environment.
How much data did you managed ?
Viewing 2521 - 2530 interview questions