LLMs and Data-to-Text §
Highlights §
- Since then I’ve had lots of discussions with people, in both academic and commercial contexts, about whether it makes sense to use LLMs to generate texts that summarise data. Below I give a high-level summary of my current views on this topic. (View Highlight)
- The latest versions of GPT and LLMs have improved a bit, but they are still not very good at analytics and extracting insights from data. And perhaps we shouldn’t expect them to be good at analytics; after all, they are language models! (View Highlight)
- the best approach is to do analytics and insight creation separately (outside the LLM), and then provide these insights to the LLM as part of its input data (View Highlight)
- I think asking LLMs to do analytics fundamentally makes little sense, but its certainly possible that LLMs will get better at discourse-level issues and hence generate better long-form texts. (View Highlight)
- LLMs are very good at microplanning and surface realisation, at least in academic leaderboard contexts. However, there are some important caveat about real-world usage. (View Highlight)
- LLMs currently don’t do nearly as well at document planning, but perhaps this will change over time. (View Highlight)
- LLMs are poor at signal analysis and data interpretation, and it is a mistake to expect them to do these tasks. (View Highlight)
- Of course in the real-world we don’t need to be purists and insist on 100% LLM solutions, we can build systems which use LLM technology where it makes sense and other technologies elsewhere. (View Highlight)