Detoxing From Social Media & Spotify’s Speech Emotion Recognition AI

05.20.21
Detoxing From Social Media & Spotify’s Speech Emotion Recognition AI (Artwork created by Brigido Bautista)

In the last of three special Mental Health Awareness Month episodes, Adult ISH co-hosts Merk Nguyen and Nyge Turner explore the relationships young people have with their digital devices, and how they can affect our mental health – for better or worse. First, the dynamic co-host duo unplugs from their screens in a strictly monitored social media detox, yielding surprising results. Then, YR Media’s Emerging Technology Instructor Marjerrie Masicat sits down to talk about Spotify’s latest patent on speech emotion recognition technology, which may be able to gauge its listeners moods and recommend songs accordingly; big thanks to Lissa Soep for her oversight on this segment. Be sure to follow our socials @yrAdultISH to stay connected!

And if you're looking for some mental health resources, we've got some for you:
Latinx Therapy
Therapists of Color
Asian Mental Health Collective
Therapy for Black Men
Therapy for Black Girls
National Queer & Trans Therapist Network
• Even MORE resources on this Google Doc


Episode Transcript

Nyge: So, Merk, let me tell you about this little story about me as a youngin'. 

Merk: Ooh yeah. 

Nyge: So basically my phone had broke while I was at school one day. 

Merk: Oh, no. 

Nyge: I started panicking. Phone was broken. I caught two busses to a Radio Shack, bought a burner phone, put my SIM card in the burner phone, and then just used that like burner whatever, $20 phone for like the rest of the day. And then I was able to get my phone fixed later on. 

Merk: Wow. 

Nyge: But I did all of that just so that I didn't have to live life without a phone. 

Merk: That's commitment. 

Nyge: That brings me to a question that I want to ask you. 

Merk: Do I have a burner phone? 

Nyge: No, you didn't (laughs) — I hope you don't. I hope you don't. And if you do, you probably shouldn't tell me about it. 

Merk: (laughs) 

Nyge: But I'm pretty dependent on my phone. Merk, does the amount of time that you spend on your phone scare you? 

Merk: You know, after looking at my screen time a little bit. 

Nyge: Yeah. 

Merk: I average about like five hours a day. But the amount of time I spend on Maps like Apple Maps or Google Maps, I'm like, "Oh, do I really spend like two and a half hours, like just looking at stuff?" But what about you think? Your numbers are probably way different than mine. 

Nyge: Yeah, I spend a lot of time on social media. That's where like all of my time goes. I mean, it does scare me because it's like I naturally have kind of like a nervous twitch to, like, pick up my phone and immediately go to like an app or whatever, and I'll be on Instagram, and I'll be like, "When did I get on Instagram?" Or I'll be on Snapchat or like on Twitter or something and be like, "How did I get here?" And that is a scary thing, when you're just like subconsciously doing something and doing something that's like so involved. I had to pick up my phone, unlock my phone, swipe over, find the app, click on the app and then start scrolling. So it took a lot for me to get there. And I did it like on autopilot. It's pretty scary. 

Merk: Well it’s those very fears that prompted the creation of this “Digital Dependency ISH” episode here on “Adult ISH!” What’s up everyone? This show is produced by YR Media. And on it we talk about Nyge and his burner phones and the fears he has of him not living with his phone. I’m Merk Nguyen. 

Nyge: And I'm Nyge Turner. Today, we’re exploring the relationships we have with our apps and our smartphones. How much do we rely on them? What does this say about us and how does it make us feel? Because like I'm not about to burn my smartphone and live in the woods or anything like that. But I mean, these things are going to keep progressing. 

Merk: But, hey, we were able to survive in those prehistoric days before 2007, before iPhones were introduced and everything like that so we could do it. But we know scientifically and like personally that all this dependency on technology can come with negative side effects, especially for our mental health.

Nyge: Effects like depression, anxiety, self-loathing, or like constantly comparing yourself to others. That's why we're talking about this topic today. So later, we're going to bring on our coworker Marjerrie Masicat to tell us about AI technology that Spotify is going to use to improve our moods, allegedly.

Merk: Allegedly. But in the meantime, our producer, Georgia, wanted to challenge Nyge and I to get off our dang phones and take part in a five-day digital detox. So she was kind enough to let us continue using our phones for calling people, texting people, stuff like using your alarm clock, setting that, using your camera. 

Nyge: Yes. Strictly no social media or any other apps that fall outside of those categories. I spend personally about 22 hours on social media a week. 

Merk: Woo! 

Nyge: It hurts to even say that, so my goal is to get pretty much a whole day back to every one of my weeks. 

Merk: Yeah, I don't spend as much time on my phone as Nyge. I'm like, you know, maybe five hours a day, if that. Earlier this year, I turned off my social notifications, which helped out a lot. But I wanted to see if me cutting out completely would improve anything, especially as relates to my mentals. 

Nyge: So, here we go!

[Music Break]

(audio montage plays)

Merk: Okay, today's digi detox was pretty alright. I did quite a bit of driving without Maps, which required me to look up where I was going to go ahead of time, write it down on a note card. So now I know how to get to Home Depot and my cousin's house. 

Nyge: Alright, fellas, it's day one, it's going cool. I mean, it was going alright besides the fact that I kind of forgot that we were doing a social media detox. Merk texted me that we were doing it, but I mean, maybe I'll just make it up on the back end, but right now I just deleted all my social media apps, Twitter, Instagram, Snapchat. So hopefully looking forward to getting parts of my life back. 

Merk: Day two. Something I forgot to mention was that yesterday I did open up Snapchat out of habit and then I realized, "No, no, don't do that." So then I deleted it from my phone. I did not delete the other apps because I don't go on them very much like Nyge does. But today was good because I already had the note card written. I was able to navigate to my cousin's house and we went out to eat at a restaurant. Did I navigate on my Maps? No, I did not. Did she? She did. But that's not cheating. 

Nyge: Alright, fellas, so day two, it's not too different. It is like a little weird because I'll go on my phone and like, as soon as I got on my phone, I'll be like, "What am I on here for?" Like, it's pointless. Like, I just naturally, like, click on my phone and I'll be like, "Okay, I'm about to like go to like a social media app." But I mean, I can't. So like, as soon as I get on there and I see like the apps aren't there, then I'm just like, on my phone. Like, what do I do? The days do feel like I have like a little longer but yeah, not too crazy, but I mean it's trending positive so I'm feeling it. 

Merk: My cousin is a social media queen, and I asked her what her numbers were at, and she's like, "Oh yeah, I spend like 10 hours on social a day." And I was like, "Maybe after this detox is when I'll get on TikTok and be connected with people who were like on that and like be totally on my social media game." But then my mind went to, “There is so much content to consume. Like I want to be able to consume everything, but that would be unhealthy and that's just impossible and impractical.” And yeah, the anxious thoughts were definitely kicking in. 

Nyge: Day three! I don't know why I did that. You know, I put my phone on Brandi's side of the bed, like at night ever since we got married. So like I think since I've been doing that, like, I'm not, you know, just scrolling aimlessly at night anymore, so that's cool. I did go out today and, you know, I was in public and everybody, you know, was like standing around in public on their phone was like looking down and stuff like that. And if you're, like, not on your phone in public and you're just like looking around at people with no phone, I never knew that you kind of stand out and like people are kind of looking at you like, "Bro, where's your phone?" I don't know, it feels like I kind of like woke up. I could see “The Matrix.”

Merk: Alright, so it looks like today I spent more time on my phone compared to other days, about two hours and 45 minutes. A lot of that was me on Messages. And I think I caved by using Venmo because that's not one of the apps that's listed on our can use list. So yikes. But ya girl had to pay for therapy and bills or you know Venmoing my roommate. 

Nyge: I did play a game on my phone. I don't think that was within the rules. So big L. 

Merk: I also had to make sure I was not using Maps, even though today was a day that I picked up my dad from the airport. So I wrote down my directions and I took a test run before actually going to pick him up because he wanted some specific food on the way. He wanted a Yoshinoya beef bowl with chicken, and then I couldn't find the airport. So I actually went back home, wrote down more directions and then navigated my way there. 

Nyge: Day four of the social media detox. I don't know what people are doing. I don't know what my friend is doing. I do wonder. I was talking to Brandi about that. I was like, "Yo, like, I wonder like what people are like up to." But she was like, "That's kind of how I feel like all the time." Like it's alright, but yeah, I think that it is going well. I have like a lot of time on my hands, I've been like running a lot, getting back into my workout grind and that's cool. So it's going good. I'm liking it so far. I'm digging it. 

Merk: Okay, we are going on the road to Southern California and you're going to be my GPS? 

Merk’s Dad: Yes. Yeah, I'm here. It navigates to Vietnam Town and we can eat some traditional food. 

Merk: Woohoo! 

Merk’s Dad: Okay.

Merk: Alright.

Merk’s Dad: Okay, see ya everybody.

Merk: Bye.

Merk’s Dad: Bye.

Nyge: It's day five. Everything is pretty cool. I still feel like I'm like getting on my phone for no reason. Like I pick up my phone, open it, unlock it. And then it's like what did I do that for? I used Maps today. So took an L, so I think that's my second L. Social media detox two, Nygel zero. I also got on like Safari because I was watching something like on TV and I was like, "Where do I know this actor from?" And then like I was like Googling stuff or whatever. And I didn't realize like I Google a lot of stuff, like when I'm watching TV or watching like videos or whatever. Like I'm always Googling, like, where are these people now? What are they up to? All that type of stuff. So that was an L, so took two L's today. Social media detox three. My bad, calculations was off. Sorry, y'all, try my best. 

Merk: Okay, last day of the detox and I think I did a pretty good job. I spent about two hours and 15 minutes total on my phone and a good chunk of that was when my dad and I switched driving. I was just pulling up some Spotify and streaming is allowed for this experiment. So that was nice. And then when we drove around more, my dad's friend drove around. So I just was trying to be in the moment, being with people IRL and I had a lot of Sudoku to keep me company. I honestly don't know if I'm going to be using my phone that much more, even when the detox is over. So we'll see. Wonder how Nyge is doing. 

Nyge: It's going pretty good. I'm pretty cool. I'm excited to get back on social media. I'm not going to fake, like I have this, like, itch to see notifications and then people like talking like, "Oh, yeah, because you saw so and so doing this, you saw it." And I'm like, "Nah." And then like, I'll be like, "Oh yeah. I'm doing like this social media detox." And then they're like, "Oh like for what?" Like people kind of talk to you like you got like you really going through it if you are on social media. So that is kind of weird that I have to like justify for work, for people to be like, "Oh, okay." Because you know, I thought you was wildin'. Like, nah. I think it's a positive effect not being on social media as much, like being off of it. You're going to feel a little better. But you also will miss some people and have some serious FOMO and be like, "Yo, what is going on in life? Like how do I communicate with people?" But I think that's like the conditioning of today. And we've got to stop that. You know, like if you think about people and you wonder how they doing, text 'em, like, "Yo, I'm thinkin' about you. What's going on bro? Like how are you doing, how you living?" So check in on your people. Talk to your friends. Don't spy on them through these apps, just let them know what's up, like a human being.

(audio montage ends)

Nyge: Okay so we heard little bits and pieces about how this detox went for you, but what did you learn about yourself during this time? 

Merk: I learned that I really do rely on my phone for navigation quite a bit, even for places that I already know how to go to. 

Nyge: Right. 

Merk: I think that's one thing I learned. Yeah. What about you? What have you learned? 

Nyge: I feel like, not even just about me. I feel like I learned about like everybody. I was really looking at people, like waiting in lines and just like everywhere that I was going and not being on my phone. And everybody was just so into their phone and they were like, looking at me without my phone, like, "Bro, what are you doing? Like, where's your phone, bro?" And that was, that's unsettling. That I literally have to be on my phone to feel like a normal person or to feel like I'm not making other people uncomfortable. Something wrong going on in society. 

Merk: That's really interesting because I feel like I intentionally would not be on my phone because I didn't want to be perceived as someone who was like a young person who's on their phone even before this detox, honestly. Granted, I spent a lot of time during this detox with elder Viets. 

Nyge: Yeah, this is going to sound like really like Gen Zish. And so, you know how much, like, it pains me to say it, but I don't know, I was thinking I was like, "Yo, is this like where I'm supposed to, like, be meeting strangers?” Like I was in the line and stuff and people would like, look, and we, like, make eye contact and then like make eye contact again. And I'm like, "Am I supposed to, like, ask, how your day’s going or something? Like something, are we supposed to talk?" And this is really strange because I think there's like an obvious answer, like, yes, it's another human, why not speak? But I don't know, I'm just so used to not. So I'm just like, "Uh, I'm not trying to talk to people." 

Merk: I mean, there also is the whole we have been isolated for about a year and slowly introducing ourselves back to people. 

Nyge: Yeah. 

Merk: How much of your life did you get back in this experiment? 

Nyge: So I was what? I was at I think like five hours or something like that a day. And then I went to like two hours or something like that. So I mean, when it went down to the percentages, the percentages said that I was down 46 percent since that original week. And then I've actually pretty much stayed consistent since then. 

Merk: Dang, that is so good. It's a new Nyge!

Nyge: It's definitely a new Nyge. 

Merk: Yeah. 

Nyge: How about you?

Merk: 52 percent down from what I was on. Again, I'm like, "Oh, that's so ..." I'm still kind of baffled by how much time I was spending on it. But like, if a lot of it was Maps and Messages, I'm like, "Okay, that makes sense." What amount of digital dependency do you think is healthy or just not too unhealthy? 

Nyge: Hmm. What amount is healthy? I think about an hour, hour to two hours a day is is healthy. Obviously, it depends what you're doing for those hours. And if you're doing it all at once or whatever, but spread out, an hour to two hours on social media a day, like you're just staying plugged in. You hitting up people that you like in a couple of baby shower-like things, some marriage, engagements, and some graduations. And it's like you looking in, all my people cool. Everyone doing okay, show a little love. And then, you're out.

Merk: Yeah, definitely putting a time limit on it I think is a good thing. Yeah, all in all, I'm really glad that we did this because it was just nice to be re-plugged with the world, unplugged from the phone. 

Nyge: Same, I've always clowned people on social media for taking like breaks. But now I think I'm a social media breaker person. Y'all are on to something and I was late to the party. 

[Episode Break]

Merk: Welcome back, everyone. We're here to chat with our coworker, Marjerrie Masicat, YR Media's visual designer and interim emerging technology instructor. She's going to tell us about some freaky new technology that Spotify might be using sometime in the future. So welcome, Marj, here to blow our minds away. 

Marj: Thanks for having me on. 

Nyge: It's good to have you here. So on this episode, we've been talking all about how young people depend on our phones and how that dependency can kind of be terrible for our mental health. But you've been researching this new technology from Spotify who says it could actually help boost our mood so maybe we could use our phones for good and not evil finally. Could you give us, like, a super broad overview of what this technology would do? 

Marj: Yeah, of course. So Spotify earlier in the year had actually been granted a patent. I think it was January 12th, so very, very early on in this year. And the patent specified that they would be using speech recognition technology, which is a kind of artificial intelligence that relies very heavily on machine learning to identify your moods and your emotions in order to shape song recommendations accordingly based off of your mood. 

Merk: How do they even recognize your mood, though, just by analyzing what it is you're saying on a daily basis? 

Marj: My hunch is actually that they would be analyzing the audio peaks and sounds of your voice rather than actual words that you are saying. 

Merk: Oh.

Nyge: That's interesting. Like I be crying when I'm like laughing so hard sometimes. 

Merk: (laughs) 

Nyge: Like would it get confused, like, you know, we don't know what's going on with you, with your emotions right now. 

Marj: Yeah, I think that's the important distinction. So Spotify is playing around with speech recognition technology, which we're all familiar with, you know, with Google and Alexa and Siri. We're like, "Hey Siri. Hey Google." And that kind of technology has been existent for a very long time. But now Spotify is specifically trying to be using something called speech emotion recognition technology, which is fairly, fairly new, and it hasn't necessarily seen too many strides in its complexity. So when I talk about speech emotion recognition, you did just have that question of, “Would Spotify be able to tell the difference between when I am crying, sad crying versus laugh crying?” And my guess is probably not currently. And the way they are actually trying to train their technology model is actually using lots and lots of data, audio data, of course, and they are providing it to others to help identify like audio. So someone who is like working with this data, they'll listen to a piece of audio and they'll be like, that person is laughing or like that person is happy just based off of this kind of sound or this person is sad, based off of the crying or based off of just like how low their voice is. And that's a low energy kind of thing. The data itself is going to be measured in a biased way because whoever on the other side is marking the audio, that in itself, emotions are not as straightforward as a label. 

Merk: Sounds like they're listening for context of whatever audio is being played, right? 

Marj: Yeah, context. Another part of the patent itself is they're trying to use speech recognition technology to also identify the location you're at. So if you have given your phone permission to just be listening in on you or whatever, perhaps they can identify that you are in a crowded bar and the energy of the bar will also contribute to sort of like the recommendation that they give you song wise. So it's like, "Oh, you're at a bar." They identified that. You sound a little sad here. Here's some music kind of thing. And so not only is it trying to identify your emotions, but it's also trying to recognize locationally where you are, because obviously, if you're in your room simpin', it might be completely different in terms of a recommendation, you know?

Nyge: Yeah, that's a little invasive, don't you think, to do that with all that data? 

Marj: I totally think it's invasive. I for one do not ever use Siri or Google or Alexa because I feel like once I start talking to a machine as if it's a person, like we're not homies like that, you know what I mean? We're not friends. I don't want to like...

Merk: I definitely have said thank you and please to Siri. 

Marj: Personally, I opt out on all of those kinds of technology because I don't necessarily want to be contributing to the growth of it. Obviously, it's become more and more prevalent in our society. And so my biggest question is like how often will Spotify or even Google or Apple be listening in on us? I think that's always the question, right? If Spotify ends up using this kind of technology, there will be more suspicion around the use of this application. That's supposed to be fun, you know?

Merk: So are there people who have outwardly been like, "No! Do not do this," groups of folks who are just against this kind of patent? 

Marj: Overall, there seems to be a general consensus with the public that Spotify should not be trying to use this technology. But in specific, there has been a large group of musicians globally from around the world. I think it's about 180, if not more, who have signed this petition in protest of Spotify using this technology. 

Merk: I mean, I'm kind of on board with it, as it stands right now, because I'm like, "Okay, well, if this is like here to help improve my mood, then let's do it." But at the same time, it's like, okay, let's say I'm at an environment where there's ‘80s pop playing "Take on Me" by A-Ha for example, and let's hypothetically say I wasn't a big fan of the song, but it's like playing on repeat because my friends are liking it. What if Spotify then made the assumption that, "Oh, you really like this song, huh?" And it makes that assumption of like they think I'm this age or this gender or race or whatever. 

Marj: Yeah, I think that's the hard thing that I don't particularly understand about this technology, specifically about the gender portion of the recognition technology, because gender itself and this is the opinion that I have for myself, of course, I feel like gender is not binary. It's not just female or male. There are a lot of nonbinary folks. There are trans folks. So how could you be labeling someone based off of either their voice or the actual music that they like? How can you actually assign them male or female? And I think that's what I don't particularly like about the the technology and the categories in which they try to assign you to. 

Merk: Pigeonhole you. 

Marj: Because I mean, I identify as a cis female, and I could just be really, really into something that Spotify has thought that a guy likes or a man likes and therefore be recommending me music that quote unquote, only a man listens to when in reality I'm a female. 

Nyge: Yeah, I'll think about the same thing, especially when it comes to like race, because I think I mean, robots don't get me, but... 

Merk: They're coming. 

Nyge: Right? I think that humans kind of understand, like, the nuance around like things because I was like at a kind of like club-ish and it was like R&B music all night. And then they threw on "Tennessee Whiskey" by Chris Stapleton. And that's not like an R&B song. If anything, that's I guess probably country. But let that come on in a Black club and see how it go up. Like we definitely love some "Tennessee Whiskey." And I mean, yeah, I don't think it's any way that like a phone or a robot or anything like that could understand that. 

Merk: Yeah. 

Nyge: But I mean, so far, I think I'm with it as long as it's not like annoying, like as long as they're not just like sending me a bunch of wack music, but I think I'm gonna treat it like handing somebody like an aux cord in my car. Like you get like two or three songs and if you're not doing it like by then... 

Merk: Opt out.

Nyge: I'm gonna have to take that back. 

Merk: Yeah, I'm into this for now. And I mean, even last night I got a notification on my phone asking, "Hey, would you like to enable your mic for Spotify to play road trip music for you?" And I'm like, "You know what? Just tell me what to do. Tell me what I can listen to." And like Nyge said, "If I don't like the music, then I would hope that there's the option to just un-enable." 

Marj: Actually, to your point, Merk. So you got this pop up that said, "Hey, would you like to Spotify speech recognition?” By saying, 'Hey, Spotify, open up Spotify.'" That I think, is actually really, really useful for when you are in the car. 

Merk: Yes. 

Marj: Because obviously you shouldn't be handling your phone. 

Merk: Exactly. 

Marj: I am honestly guilty as charged because the song that was on was not poppin' enough, you know? 

Nyge: Yeah. 

Marj: So I had to find something that was in my mood, but I'd be willing to turn it on for that specific use right? It's just like I'm about to hop in my car. I don't want to be handling my phone, but I want to be able to tell Spotify that I want to be listening to something smooth or something jazzy or something like the top 50 hits. 

Nyge: Right.

Merk: Well, one thing's for sure, and that's music is definitely a big way I and I think a lot of us manipulate our own moods, like when I'm feeling up, I like to listen to tropical house or Years and Years or some splash, a little Joji in there. And then when I'm feeling down, I need something to kind of lift my spirit, so like listen to worship music or this artist named Su Lee, who I really like. She's awesome. What are your mood boosting songs? And let's see if Spotify will play them in the future if you enable your mic. 

Marj: Yeah, I was not prepared for this question, but I'm kind of ... So lately, I've been listening to Almondmilkhunni. She has a song with Flo Milli called "Cherry." I have also been listening to an artist named Thuy. She is a Vietnamese Bay Area artist, and she won 106.1 KMEL contest a while ago. She has a song called "Vapor Rub." And I think the last person that I've been consistently listening to is IamDoechii. 

Nyge: My number one, like, feel good song, especially when, like, I originally was going through, like learning how to manage my anxiety was like the only thing I could play that would make me feel better was "Nothing Even Matters" by Lauryn Hill and D'Angelo. And I would always like play that. And it would — by the end of the song, I would always feel like at least a little bit better. And then I guess a kind of current one would be, "I Want You Around" by Snoh Aalegra.

Merk: I think my top two are "Hey Doll" by RuPaul because you know 2021 has been a very interesting year in terms of mental health. Like I thought for me it was going to be 2020 but 2021 was actually worse for whatever reason. So that song just reminds me like, “Hey everything's going to be okay.” And BTS' "Answer: Love Myself." Again, it just kind of plays into the, "Okay, if you're having a hard time nurturing the soul that lives within you, let BTS just kind of lift your spirit." And it's the song that was recommended to me by one of my best friends. So it's like having my best friend recommend that song to me versus just a playlist. There's that human connection that's there. That memory. 

Nyge: Gotta love some human connection. 

Marj: That obviously Spotify won't be able to give us if they use this patent.

Nyge: Facts. Well, thank you for keeping us aware, Marj. If you want to learn more about this technology, y'all can go check out Jay-Z beefs with AI and Erase Your Face on yr.media. And if you want to learn more about Marj and her work, you can go to her website at www.marjcat.com. That's M-A-R-J-C-A-T. dot com. 

Merk: Thanks, Marj. 

Marj: Thank you. 

[Music Break]

Merk: So top takeaways are one: For most of us, some form of digital dependency is unavoidable. However, more growth than you think can come from disconnecting with that little robot you got in your pocket.

Nyge: And two: we should be aware that change is coming to our technology, but only time will tell whether that change is good or bad.

Merk: So with that, thanks for listening to Adult ISH, produced by YR Media, a national network of young artists and journalists creating content for this generation. Special shoutouts go out to our producer Georgia Wright, Executive Producer Rebecca Martin, and the young people at YR who contributed art for this episode. 

Nyge: When you do spend time on your phone again, make sure you’re following us on all the socials @YRadultISH or come to our site adultishpodcast.com. 

Merk: We’re also proud to be members of Radiotopia by PRX. An independent listener-supported collective of some of the most dependable shows in all of podcasting. Find them at radiotopia.fm.

Nyge: And now for next week’s out of context clip.

[growling of a panther]

Merk: Can you give us any clues about that?

Nyge: Uhh nah, just wait and see. It’ll be fire.

Merk: Okay ... until fire. 

Nyge: (laughs) Later.

Support the Next Generation of Content Creators
Invest in the diverse voices that will shape and lead the future of journalism and art.
donate now
Support the Next Generation of Content Creators
Invest in the diverse voices that will shape and lead the future of journalism and art.
donate now