Using the AI as a tool, what happens to programming languages and IDEs? 

LLMs have an intrinsic problem with accuracy. As a result, their ability to write correct programs is limited. However, they are useful in providing sketches/templates or completing boilerplate tasks. Moreover, it seems that incorporating them in a suitable enclosing architecture may help address these limitations.

It seems that the best applications of the technology are where text is hard to write correctly, but relatively easy to verify. Generating programs doesn't really fall into this category. It is true that programs are hard to write, but they are also hard to verify.

We need systems that are easy to verify, even if harder to write.

In general, systems that are difficult for most human programmers to work with, but quite easy for machines to verify, become more attractive. Pure functional or logic programming might be beneficiaries.

As an example, consider the Hindley-Milner type system.  What if an AI could clearly explain how to fix the type errors; better yet, what if it generates programs and ensures they are typesafe, by repeatedly interacting with the compiler so you don't have to? You can then read the resulting program and see if you approve, but you don't have to deal with the types.

Rust is yet another language where the typechecker is difficult to work with, but offers strong guarantees.

The immediate and obvious step is the integration of AIs into applications like Office, Google Docs and so on. Conversely, we see the integration of applications into the AI interface via plugins. The latter is more significant.

In general, we can see that there is much less need for elaborate UIs in the traditional desktop/WIMP mode. To an extent, we revert to a command-line interface (CLI) - except that the commands are in natural language, and need not be limited to a line. This has enormous flexibility.

Going further, even text may become less significant as audio takes over (say, via earbuds with microphones one whispers to).

There are situations where a UI is still preferable. If the queries are spatial for example: it may still be easier to point, gesture or otherwise manipulate an analog control like a slider. A UI may also be useful for repeating some action.  

However, complex hierarchies of screens and/or deeply nested menu structures seem unnecessary. In cases where a specific UI is needed, it can be generated by the AI, and customized to the user. Specialized apps will be replaced by the general chat interface, or by custom generated, simple one-off UIs.

We can expect the UIs that distinguish operating systems today to become less significant, and the OS' commoditized, as control of the device is increasingly done via chat.  

This is much more interesting. I expect that we reach a stage where we describe our requirements in natural language and the AI does the rest. We aren't there yet, and even if we get there, this will be a process of socratic dialogue.  Natural language is inherently ambiguous. Either the AI asks clarifying questions, or else we get a result and play with it to see how we want it changed. So this isn't always an instant process; it still requires work, it's still programming, but in natural language.

Programs don't go away however. AI is a very costly way to compute, so whenever possible, we want the AI to write a classical program to do the computation. 

Ease of customization is a key point here. Natural language is the ultimate answer to the problem of end-user programming. It is the true "no-code" solution. For the first time, non-technical users of computers can have complete control over their digital environment.

The real significance lies in the democratization of programming.  Now, everyone can control their digital environment, not just people with special training and a strange aptitude for thinking like a machine.

In this scenario, the AI is a personal assistant, separate from apps, so it can run, customize, program and integrate multiple apps.  Ideally this AI is on the user's machine, but if even is a service (say, because it doesn't fit), it is the only non-local service one needs. Ideally we want a local-first AI, which can run locally, perhaps at diminished capacity.

Clearly, it isn't enough to have a fixed set of extensions, or plugins, or customization options.  We need to be able to reprogram the system/document (via AI), so it must be open source. 

Moreover, it is crucial that we have the document be live. All the arguments in favor of live programming still hold. We want to provide people with instant feedback on their changes and we want to preserve the state of the UI and the computation even when the logic of the program is modified. If anything, this is even more important now that the "programing" is done by ordinary people rather than by trained programmers.

Of course, one still wants this document to be shareable and sync'ed across multiple devices. In fact, one still wants all the properties I argued for in my previous post.