Language models on their own do indeed have lots of limitations, however there is a lot of potential in coupling them with other types of expert systems.
Still, while keeping this in mind we also must remain mindful of what each system can actually do, not conflate with what we WANT it do yet it can not do yet, and might never will.
Absolutely, I even have a dedicated section “Trying to insure combinatoriality/compositionality” in my notes on the topic https://fabien.benetou.fr/Content/SelfHostingArtificialIntelligence
Still, while keeping this in mind we also must remain mindful of what each system can actually do, not conflate with what we WANT it do yet it can not do yet, and might never will.