There's nothing aspiring about what you wrote. I think you're fine calling yourself a data engineer if those are the types of challenges you've been solving.
Don't sell yourself short or select yourself out of an opportunity (within reason). That's someone else's job!
I love data engineering - atleast when I practiced it at an adtech company, it involved lots of interesting challenges around performance, distributed systems and a mix of SWE & Ops work. Buut, applying to data engineering jobs I see where the parent poster is coming from (minus the condescension) - a lot of them are about writing simple ETL jobs, and they tend to be lower paid & less prestigious.
The skills you described are those of a data scientist/engineer hybrid. Have you tried clearly branding yourself as a data engineer? There’s a lot fewer of those, and the job is overlapping to the nearest understanding of a hiring manager or non tech person.
It sounds like you'd very much enjoy working as a data engineer. Data engineers are actually more in demand than data scientists in most place right now, so you're in a good position if you choose to go down that path.
I think the idea of a data engineering roadmap is flawed because there isn't a single cookie cutter "data engineer" role to aim for, just like if wouldn't make sense to talk about a single route into becoming a "software engineer".
I'm a data engineer but spend most of my time writing code in dask, and use fairly little of SQL. I've been in roles before though where long and complex SQL routines was the main focus.
My advice that nobody asked for: focus on 1) general programming skills which are transferable anywhere and 2) do what you personally find interesting. Like SQL? Sweet plenty if people will pay to do that. Obsessed with using scala and spark in combo? Someone'll probably pay you to do that too.
Don't focus on being a "data engineer" or any other label- just enjoy programming and find roles that have more of what makes you happy.
I've seen a disturbing rise in the number of people who think data engineering isn't software engineering. I don't plan to play up that part of my experience the next time I'm applying.
As an undergraduate who is about to graduate with a degree in "Data Science" this post encapsulates a lot of my worries as I move into the work world. Should I focus on being a "thinker" a "doer" or a "plumber"? For the first three years I was planning on being a CS major until I was denied from the department: now the data science major is my only hope to graduate. I feel as though my programming skills are solid: but not good enough to be on any sort of fast paced infrastructure/devops team. On the flipside: I feel as though I am so far behind on stats/math knowledge that it's pointless to try and become a data scientist/analyst. I've thought about data engineering (the 'doer') as a happy compromise between the two. However there are barely any intern or entry level data engineering positions that I can find. The ones I do find require knowledge of so many frameworks that I don't know where to start. Additionally, I'm not even sure if data engineering even is a happy compromise, especially after reading the post. Time is ticking, and sooner or later I'm going to have to figure out what route to take, and how I want to specialize. I go to a hyper competitive university in a hyper competitive region of the country and I'm starting to feel like I'm falling behind and getting lost.
If any of you older/more experienced engineers and scientist have advice or wisdom for me, I would very much appreciate it.
I've been a data person for the past year and a half and I'm very disappointed with the bewildering array of titles out there and the rather vague meanings behind them (Data Analyst, Data Scientist, Data Engineer, ML Engineer).
It's overall hurting my ability to build my personal brand and seek roles that are a fit for my existing skillset and aspirations.
What exactly does 'ML Engineer' communicate to employers in terms of baseline skills? Is the role closer to that of a data engineer or an analyst?
> Data Engineering sounds interesting but also appears to require heavy statistics.
That's Data Science. Data Engineering is just writing code (pipelines in Spark usually) which moves data from system A to B and perhaps applies some machine learning code, supplied by data scientists, to it. But, most of the work is just moving data around. I work with dozens of Data Engineers and none of them know even basic stats (and if they know, they don't get to use it anyway).
Don't sell yourself short or select yourself out of an opportunity (within reason). That's someone else's job!
reply