Yes, fully agree it is funky and not even close to being intuitive to understand or to get dialed in.
I can relate to your frustrations, I went endured the same thing. For months, out of nowhere, warm air would start blowing into the cabin. I only want fresh, outside air to flow through, no heat, except on really cold days.
My approach was to start with everything off, including the S-Flow setting and the so-called Climate Concierge. Sort of a base case starting point where nothing is in Auto mode, the fan is off, the A/C is off, but the Outside/recirculate air function is on (is allowing in outside air). Be aware of the two or three possible modes for every button on the console, indicated by 1) white bar, 2) blue bar or 3) no bar, IMO these are confusing and mixed up but we have to live with them as is.
See post #20 in
this thread. I've watched many times over the NorthSide Lexus video at the timestamp listed in the thread, and I'm still not totally sure I have all the different settings the way I want. And then when I do get them the way I want and think they are all set and all dialed in, I glance at the center console and one or the other has changed from a white bar to a blue bar. Usually the Outside/recirculate air function. It's very frustrating.