Dear Forumer in Lowyat,

I am sincerely would like to ask for opinion from all you on what do you think if I am going take a Master in Data Science course.

My background:
1. 33 yo, Chemical Engineering degree, 10y working experience as process engineer. Previously work in plant now work as process specialist for a European plant machinery supplier, considered having higher than average pay but travels for project commissioning really a lot, 60~70% days per year on average (and this year is 0% travelling due to covid ban)
2. Passionate in mathematics, logical thinking. At work because I am able to read and recognize patterns in process data that most people ignored -- given me a leg up at work.
3. Able to do simple VBA quotation for looping / automate some excel calculation procedures for data, understand basic C++ programming, not expert but I am confident it will not be a major challenge for me
4. Other subject of interest: economics/psychology - manage own investment portfolio, study statistical movement of markets etc. Only as interest.

As an engineer from industrial background, do you think it is worth for me to go into Data Science? May I ask if anyone know how is the Data Science job market in Malaysia? is it mostly concentrated in Banking, ecommerce and logistics and that is the best place for Data Scientist to work in? Manufacturing Data Science / IR4.0 etc is practically exist / soon to be or just theoretical? Are they exist in oil and gas / processing plant etc?

My reason of thinking about taking the this course:
1. This subject really really interests me 50%
2. Market demand 25%
3. Always thought of further study 10%
4. Wanted to stop traveling job for various reason, looking for good alternative 15%

Thank you for taking your time reading it.. I am appreciating everyone who clicked in this thread .

IMHO a lot of people are switching into data science hence will have lots competition when you graduate....and pay wont be good as now.....i seen over thousands applicants for a data science job........and from every background....

There are some people with 20 years of experience from banks / insurance using SAS at that time was called only Predictive Analytics....and then there are tons of fresh graduates (ADAX and etc) in data science (R / Python) plus people wanting to switch because data science seems cool (due to hype) switching from customer service, insurance , bio / engineering even IT and accounting grads with all studying masters . phd or coursera yet dont know how complex it is to do data science.....u need to know the whole eco system from data (as in unstrutured n strutured n relational n non relational n etc, statistics, programming and most important subject matter (marketing, risk, manufacturin, ai, visual, hr or....)

the work of a data science varies from industry to industry.....in a factory it might be used for predicting machine failure or faulty products.....and in real estate to predict property price.....the alogrithims and how you get the data will vary as well......some u need yo get data from unstructured sources and some from databases/store and databases/store have many types - rdbms, nosql, hadoop and then how to transform the data to your prediction model (poisson? / binomial / exponential, normal).... u will need to know stuff like json, hdfs, sql, phtyon, r and how to join those datasets .and bloody more....the risk of automation of your job with the rise of Auto ML..

Data Science is a tough life.....and with many competition now. (like aspiring actor / singer)....pay has dropped drastically.... most asiprants calls them self and think they can do data science without even knowing at what level they are performing.......with most boss and organizations have different idea of what a data scientist is and can do........the worst expectation is 1 data scientist can build all AI stuff without a team.....met an low cost airliine Chief Data and he aspirationw as to build an algorithm to define the whole airline and he was a reputable startup bur never had any programming or data backhkground..and a CTO of a Telco M that wants to use data science to build AI to replace CEO and management as he feels data / AI is better than humans.........lots of hot air......data science is nots just math, not just porgramming and not just reporting......and definitely not just an AI that can talks to you like human do

We have engineering PHDs, Business PHDs, even Real estate PHDs doing data science yet never know how to code before............just talk.....and with automated machine learnings/data sciences tool not sure we need so much data scientist in the future....even if need probably be like current engineers / SAP consultant salary not gonna be high pay unless you can be a real unicorn ie full stack data science.....knows everything in data tech, programming stats and business and make sure you keep up when it evoles......

so yeah currently every tom dick harry ahmad mutu sammy wants to be a data scientist...becasue the have some wrong impression of the work and future pay....

better do a doctor...or lawyer or accountant if you are starting out only now.....pays higher and more secured work......and probably next few years data science is just a tool like excel.......any normal person can just use without programming or understanding the inner works.....

similarly in the 80's everyone wanted to be an Engineer / Pilot.......now engineering / pilot role is not as lucrative as Marketing and Sales......

i feel data science and digital will have web designers / DB Administrator fate in the future........unless u really stand out......new and cheaper person or automation(web templates, word press, what you see if what you get tool for exmpl) can replace u

oh yeah thing is that is funny to me ....most data scienctist can come out with a model ie regression, classification , nueral net to predict test cases.....but how to prodcutionalizxe it in real world clueless......for example.....how embed those predcitions into your system...ie example (grab app for predicting traffic, netflix form recommendation of films, predicting fraud when someone tries to do a transaction on online banking)...you cant simply run/embeb R or SAS or Python in IOS app or C++ solution.......which will pull real time data from your data sources...food for thought..

This post has been edited by buffalowings: Nov 13 2020, 10:31 PM

Wow that is a very detailed view on the prospect of this career, that you so much for telling all the good and ugly side

I get what you mean about the how crowded it is now a lot of DS could be not having adequate experience about the sector itself, and the uncertainty on evolving of this field itself... I guess that really had influenced me how I should weigh this thing, appreciate your opinion so much

I agree to a certain point of what bufflowings had said. But let me give you my twist.

I was from a mechanical engineering background, and worked n the automotive industry overseas. When I came back I switch my job to be first a data analyst, and now a Business Intelligence analyst. I am in my late 20s. I have found that data science or data analyst is definitely more a buzz word that what it is used now especially here in Malaysia. Most companies are not at the level yet were they truly require predictive analysis. The advantage for you and I is that since the field is so young, going into it now would give us a leg up in the coming future.

Being a data scientist/analyst is not just about doing modeling or prediction. But the first step is to collect the data, understand it, drive insights from the data and the soft skills aspect of it to be able to convey and present your insights with data across to management. Not only for the purpose of understanding, but also for them to act upon it.

So to answer your question, I would say there are several ways of entering the field of data, it doesn't necessarily have to be a master's degree. And if you are thinking of taking an only local masters degree in data science, I would urge you to forget about it, since they are so new and do not really know what they are teaching too. It would be better to learn on the job, or taking courses online and try to apply it on the job. Soon enough you will find yourself out growing the data that your company has if hasn't undergone digital transformation, but you could use that skillset and knowledge that you have and apply to other roles outside of your company. Hope that helps.

data analyst, business analyst, business intelligence analyst, finacial analyst bank analyst, database programer, statistician, market researcher , sales analyst and many more thinking they are data scientist and assuming data science is just about understanding data or extracting data or doing SQL or putting data in nice graph for management to understand using Power Point ......thats why the market is flooded....a data analyst is imho is way far from being a data scientist...its like someone who just started learning how to speak claiming can publish a song album ....hahaha.....not to butt hurt anyone .....but if you dont know how to put together a solution that makes use of technology to process data via statistical modelling or ML ...you are just...a normal analyst..and not a data scientist.....unfortunately even organizations have diff thoughts wat a data scientist role is so good luck....know a telco GM who never done tech and from a sales background....was head of data science because he claims he understands data.....hence anything else can outsource to engineers...the most important thing is that he can interpret data better than less experience person.....(wat a joke)

a data scientist should know how to implement a real world data product and not just to story tell about data (this is like basic)... for a simple example......how to build a recommendation engine, how to create a credit scoring system or how to build a system to detect data phising ........and not just to report sales trend or answer what was my top selling product, which customers bought it and give a recommendation how to improve those reporting insights? (this to me is just analyst role using descriptive analytics basic skill to have)........as a data scientist you would know how to build a working prescriptive analytics solution end to end with constant data input and output.and it you are real good a solution that can slef learn......and nearlly 99 percent of the time...its a solution where a human cant interpret the amount of data that need to be analyzed....hence why is called big data.....for example netflix engine goes thorugh a billion customers viewing behaviors via clicks and interaction with thier product......and then use a statitcal model to find similliar patters for up to >50 movies recommendation for 1 person...similiar with facebook updates you see...they are personalized to a single individual based on algorithms that read billions of interaction the user of the world have with facebook.....a human analyst would not be able to interpret it........hence why you need data science........and data scientist that build data science products.......not a just hjire some one who can find data and interpret via excel.....or BI......that IMHO is an analyst......and you dont need a Masters for it....just go have on th job experience but please dont think u are really doing data science.......you just lying to yourself by having an inflated title....

another good example is alpha go which uses neural nets to study millions of players moves and then come out with a solution to beat a human player as well as IBM Deep Blue for Chess...the solution has to real time and via each move the human makes it predicts what the best move to make and do it real time.....based on its experiences from millions of possible combination.......a human wont be able to process such vast amount of data....if its possible then we only need an Analyst to give insights to the CHess / Go player in Powerpoint for wat should be the next move......lol

Hope the above shares why data science can be such an exciting and well paid job if ppl understand what it should and can do.......

and why if use just for analysis, insights and post event action recommendation by humans there is 1000 capable ppl can do this unlike the above where is a combination of technology and statistics and subject matter data (millions).....to provide realtime actions.....but like i mentioned earlier the other devil is more and more techs developers (google / Miscrosoft, AWS) are findings ways to automate the data science work using various Auto ML tools......so hence might be too late to the game .....where in the future you can buy data science off the shelf.......so probably a dat science wont get a high premium but definitely could be better than pay than an analyst....but likely be lower that a legal professional such as Accountant, Lawyer or IR Engineer where expertise in an subject is not easily duplicable by machines or humans.....

if everfyone can do it.....u wont be as valued.....if only limited ppl can do it....then u r an expert in that area and can ask for a premium in salary......a Masters. / PHD is window dressing that can open nice doors but cant help you to create the work........or maintain the role if you dont really know the job....

This post has been edited by buffalowings: Nov 13 2020, 11:36 PM

There's a lot of very useful info already mentioned by both buffalowings and joshtik1. I do agree in many ways that there's a huge gap between a BI/data analyst vs a data scientist just as there is a gap between a financial analyst and a quant.

Many companies don't have enough data, don't have a proper data science team and have no idea what to do when they have the data/insight. Also many leaders forget how important business or cross functional knowledge is in a data science team. We used to collect a lot of production/operational data in the oil and gas industry but we needed deep hydrocarbon/well engineering knowledge to make sense of it.

As mentioned already, you don't need a masters to jump into this. Data science is one of those fields where you can legitimately practice on your own. Download data from NYC open data and start experimenting!

Good job on the detailed write-up! I agree for the most part. The funny thing is, people will be posting vacancies for DA , but inside the JD is actually for DS lol. Basically the same story as as other tech jobs, where companies need to hire an entire IT department but only can afford to pay for 1 person, therefore we give him an inflated title like "CTO" or "Head of IT" and maybe give him some stock options. Now, imagine this "CTO" can't actually code but only can talk cock about fluffy concepts. That is the state of things in our country now. Much cringe.

In Malaysia, most companies already "koyak" at the data collection stage. They think they need a DS as a magic bullet to fix the whole thing but they don't. NOT YET. What they really need is a team of data engineers to solve the root problem first. DE is very un-sexy job but it is the first step that you cannot afford to fuck up if you are serious about deploying machine learning models in the organisation. So in terms of practicality if you need a job, I'd say DE is the most secure one. Without DE, literally the whole data team can die.

exactly......

Thanks so much for your insight! Glad someone like you from similar background, guess I would really take your advice, for now just forget about local master and think about equip myself with some courses to get prepared. Thanks!

Nowadays anyone who knows little bit of R, phyton, regression and looker called themselves data scientist. A good one is way more than that and takes years of research and technical expertise. Forget about local masters and if you really want a masters, I suggest to take the GA tech masters online. Cheap and reputable but please note the real learnings can only be gotten by getting your hands dirty; join kaggle, read and contribute

As mentioned by others, DS is fancy and trendy now just like web developer during the dotcom boom and now being replaced by the likes of wix etc. I believe the same will happen to DS as big tech companies are trying to democratise ds and promote citizen data scientist by creating more and advanced cognitive services through their cloud ecosystem. Unless you want to wish for tech companies, most likely what you’ll need in the future is to know how to integrate different services and bring the code to production. What still remains in the entire process is the data engineering work because as fancy as the ds algorithm, a small fked up in the data pipelines or master data your model is not reliable anymore so I always emphasis the importance of good data management. Not fancy, not dealing directly with users or customers but everyone just needs

Nevertheless, there is always someone who excels in everything they do and if you’re one of them, DS can be very lucrative because everyone wants one even if they don’t know what to do with it. Good luck !!

This post has been edited by Vinci777: Nov 16 2020, 08:09 AM

Let me direct to some links of how some of them deal with data:

https://informationisbeautiful.net/

http://bost.ocks.org/mike/

Agreed.

All the time when I go through my clients' data, the mundane job is to get them
1. collected (some have so many systems in place and can't come up with overall view of their own systems) and
2. cleaned.

Excel is still their favorite.

I truly thankful for all your inputs! Getting a clearer picture here, seems that democratizing of DS is a real threat, and local master courses are not the best option.

"most likely what you’ll need in the future is to know how to integrate different services and bring the code to production" yes that's what I am looking for as well... and data engineering seems to be another important role going to play in future even in industrial I guess?

buffalowings Hahaha wdf. It was a great opinion bro
When I saw TS mentioning about Data Science, I dunno, in my head automatically saying it's just another excel expertise but wanna have a bombastic position's name.
I'm an AI graduate. Yeah we learnt about machine learning, neural network, data analytics etc. Once we graduate, most of my friend is just a common software engineer, programmer, project manager etc. which related to the IT industry. Sometimes it is not about we don't want to implement/use our knowledge, it's just the technology in Malaysia is not up until that level.
If we want to fully utilize our knowledge, it's either we work with MCMC or similar organizations or lecturer/researcher.

Wow, VBA. Pretty rare these days
I do think VBA is a great start, and there are demands especially in the finance and shared service sector, and the pay is pretty good, at least to me.
The demands are to mostly to automate their operations processes, but you do need more advanced VBA technicals to better perform in those roles, ie connect into web apps, databases, Outlook, SAP, amongst other stuff.

Good luck!

This post has been edited by sendomike: Nov 16 2020, 10:41 PM

someone who doesn't understand about statistics/DS talking about statistics/DS lol

Brother, go back to school and get an A+ in probability and statistics and then revise your statement. Recruiters are not dumb; they can smell a fake Data Scientist from a mile away when asked to explain about concepts like Markov Chain or a Tukey test

This is exactly why companies should hire Mathematics/Statistics/Actuarial Science/Computer Science graduates only. Set a barrier of entry for Data Science.

This post has been edited by chromatino_hex: Jan 18 2022, 06:15 PM

I was a comp science graduate , graduated almost 20 years ago.

I learnt about AI back then when AI wasn't popular at all.

Then suddenly Netflix ran a context to see who can best predict the consumer's watching behaviour and Data Science flew off ( Using random forests, ensemble technique etc etc )

and everyone's calling themselves Data Scientist , Chief Data Officer , whatever fancy name you can think of and I do resonate with some forumers here who mentioned about those position taken up by pple who doesnt know a single thing about data or technicals at all.

Not to say I am a genius of sort , but I do think there are certain disservice to a true data scientist, true data engineer, true chief data officer, if you put some one who are good at telling stories without much technical or statistical background to head a team. The team will eventually implode and these true data engineers/scientist will be demoralized and company doesnt get any value of out their investments.

Having said that, some bigger companies set up the team for experimental purposes just to see what it does , or to a certain extent, bragging rights, cause they can. So its a sad stage where you see all these top level buffoons who knows how to present and sell , but technically jack sheet or mathematically idiotic just to understand basic probability concept.

Well , that is just how the industry works, and how company politics works , but the silver lining is you see nowadays , top companies are pushing more enginnering/technical background to the top job ( , ie Satya Nadella who is a technical guy , vs Steve Balmer, a sales guys, and look at the share price. Hope this decade , we can see more true engineers/technical rise to the top.

Also , i got my hands dirty to understand data science , and I realised the most basic thing is to understand statistics first, before anything else, and of course, to have deep knowledge in your domain area , then only you can truly apply what you learn in data science, to solve your industry challenges.

Agreed with everyone that most of the local data/analytics industry or departments are half baked. I'm sure there are some real good ones but rare. I'm from a Fortune 500 mnc, leading data analytics for a major function in the company. My personal experience is: for real data transformation, I tend to engage consultancy companies ie Accenture in a contracted turnkey manner (and they're mostly Indian based). I find the ability to hire locally is limited. Mileage may vary for you.

This post has been edited by Obosh: Jan 24 2022, 10:49 PM

So.... does this mean a mid career change to data science / data analyst is not recommended?

no, it is all about doing what you really want to do. No one can stop you from making any decision. At the end of the day, do you have the skills/passion/drive to make that change? Learning something new isn't easy and it would take a lot of hard work.