Yeah we can handle the part where it knows how to express itself in a specific language, it can take in some facts, compare them to its internal database and spit out something sensible as a statistically probable good reply. But there's no sense of self involved there.

I remember reading an interesting article a while ago about at least in the human case the basic principle of emerging consciousness happens when the prediction system in our brain designed for figuring out what other entities around us do is used on itself, trying to explain what the subconscious is doing. As such the consciousness we experience is a bit of a bug in that system that turned out to be beneficial to some extent. All just a theory of course given how much we actually know about the brain so far, but it's always made the most sense to me.

I'm not sure how that would translate into the current ML environment though.

Do we want that if we could have it? I want machines as slaves for me: go wash the dirty dishes and then do the laundry. I don't want it to get depressed about doing those routine jobs.

Well we can have our cake and eat it too. Keep simple and limited machines for work, and intelligent ones to talk to and treat as equals to help us in other ways. It's not like every roomba needs to run a neural inference engine to do its job, nor do we hand pollinate ever flower ourselves if we can get the bees do it it in the organic world.