Wasted 6 months training AI on bad data before I figured it out

I was working on a customer sentiment model for my company based in Austin. For months I kept feeding it forum posts and reviews, but the results were all over the place. It kept flagging totally neutral stuff as angry or excited. Last week I finally looked at what I was actually using for training data. Turns out half of it came from a scraper that grabbed comment sections full of bots and spam. A coworker glanced at my dataset and just said 'this is all garbage' and walked off. I was so focused on tweaking the model I never checked the source. Has anyone else had their whole project tank because the data you thought was clean turned out to be junk?

2 comments

2 Comments

janaw1113d ago

Did you ever check where your data actually came from before you started training?

emma_mitchell13d ago

wait @janaw11 didn't you find out yours was from some random data broker like mine?