Spark question based on transaction and user and answer top users, fraud users and other based on joining, window. LLD & optimisation of code
Senior Data Engineer Interview Questions
2,552 senior data engineer interview questions shared by candidates
Experience with DBT, SQL, Python
Technical questions regarding the different technology
The Python technical challenge was about ingesting some data in a specific way and do some analytics on the results. The questions were about the reasons behind each decision. Also, they asked for alternatives to the simple provided solution (it was a challenge done in 3h) more oriented to a production-ready environment.
I got a take home assignment to build SPARK application on recipes dataset. The problem was easy but the expectations on the solution was very high. The code should be object-oriented, with config files, unit test cases, logging and alerting, exception handling. Extra points for CI/CD , etc
General questions regarding your career up to now, along with some (general again) IT questions. Perhaps, they are competent enough to infer your knowledge without being too specific. In any case be prepared to go through your whole CV.
About my current role support activity n knowledge I have on the technology I was working on.
Questions about Dimension Modelling, AWS services, 3 sql queries and 3 python problems.
1) Kafka Exactly Once Internals 2) Spark Structured Streaming state management
How much data did you managed ?
Viewing 2531 - 2540 interview questions