Ask the accessibility expert (en): What AI can and cannot do in accessible publishing

The expert

Gregorio Pellegrino, Chief Accessibility Officer at Fondazione LIA, is an Italian software engineer with a strong focus on improving digital accessibility for visually impaired individuals. His expertise is reflected in his co-editing of key industry documents, such as W3C EPUB Accessibility 1.1, the Accessibility Metadata Display Guide for Digital Publications 2.0, and the EPUB Accessibility - EU Accessibility Act Mapping. He is also technical leader of the DAISY Consortium's "Accessible EPUB from InDesign Expert Group," working with Adobe to enhance the accessibility of EPUBs exported from InDesign. Pellegrino's work with organizations like W3C, DAISY, and EDItEUR helps shape global accessibility standards.

Introduction

Gregorio started with a poll with three questions:

Have you experimented with using generative AI in the field of accessible digital publishing?

• 35% of the audience has never tried generative AI yet in accessible digital publishing
• 50% did some experiments
• 18% uses it regularly

Are you satisfied with the results obtained? 

• 10% are satisfied
• 50% more or less satisfied

In what area have you experimented with the use of AI in accessible digital publishing? 

• Most people tried to use generative AI for image description
• Some of the audience uses it for remediating non-accessible e-books, quality assurance, checking the language of the content, metadata and for accessible production

After this Gregorio gave a brief introduction.

AI in accessible publishing goes beyond generative AI, which is only one of several techniques available. Earlier approaches like machine learning and deep learning have long been used to extract information from images and generate alternative text. Generative AI is powerful, but not always the best solution for specific tasks, where traditional ML or DL can be more reliable. A key limitation of generative AI is that it produces different answers to the same question due to built-in randomness. This makes it difficult to use in industrial workflows where consistency and responsibility for outputs are essential. The “temperature” setting in large language models influences this variability, with higher temperatures producing more creative and less predictable results. Even at temperature zero, small variations remain by design. Because of this unpredictability, it is impossible to have full control over generative AI outputs, an important consideration when applying the technology to accessible publishing processes.

Questions and Answers

What are the most promising and impactful use cases for AI in accessible publishing?

There is image descriptions, language identification –especially for language shifting–, automation of repetitive tasks, quality assurance support, as well as remediating. There is a warning however: be aware of the vendor hype that is going around. There are many claims that are inflated, telling that AI can fix everything. There is no magic button to get an accessible ebook out of a super non-accessible starting manuscript. Always ask for concrete evidence and test with your content before adopting an AI solution. Have a human in the loop, checking the results, fixing the results, because the machine alone is not always correct.

To what extent can AI generate high-quality alt text for different types of visual content, from simple photos to complex diagrams, charts, tables, and what level of human review remains essential?

Fondazione LIA has conducted extensive experiments, and the results vary significantly based on the image type and complexity of the tool used. What works well is passing the context, which means, the text before and after. It can help in getting a better response, so a better description. But you have to pay attention to the copyright issues, because passing the part of the text of the e-book along with the image to a commercial system may break some copyright agreements that you have.

But, sometimes the generative AI model is overwhelmed with the context, and then the description is more or less a duplicate of the context. Gregorio suggests to use a quantitative approach, meaning using multiple tools to generate the descriptions for the same image. Then start to edit the description from the best result that you get.

Quality control is essential, it's not something that can be run without any control. And also, there is not one tool that fits all the different types of images.

Are you seeing improvement rapidly? Are you monitoring how improvements are being implemented?

The quality has improved, even if these systems are not really trained to describe images for the purpose that we have. But it also depends on creating a good prompt that takes care of the context, of describing what we expect from an alternative text. For example defining the length that it should have, also the audience that we are expecting for that alternative text is important. And stressing again to always track it manually, because we have seen a lot of what are called hallucinations, which at the end are just bugs. So, for example, you always have to check if any text is present on the image. Is it correctly reported in the description? Check for errors in the language of the description. Maybe the book is in Italian, and you get a description in English, which is not really good. We suggest not to use fully automated alt text generation; we have seen a lot of EPUBs with alternative description generated by generative AI, without any check by the publisher, with wrong content in them that makes the whole EPUB invalid. So, use it as a tool, as a starting point, not as a magic button to fix all the e-books that you have.

How can AI support publishers in designing born-accessible content from the outset, rather than retrofitting later?

In a lot of cases the workflow is fixed as a process, of human people working. Many issues related to creating accessible digital publications can be resolved without the use of AI. Using standard tools, standard checklists, standard checks during the workflow, the quality can be improved. Gregorio does not think that generative AI contributes a lot in this process. Even with the use of AI, a lot of manual checks are necessary. But many things can be checked without generative AI. For example, checking if an image has an alt text or not doesn't require generative AI. Language checking doesn't require generative AI. Hierarchical structure for headings, meaning if the structure of headings is correctly set up, doesn't require generative AI. The presence of accessibility metadata doesn't require generative AI. The current problem is the lack of accessibility checks during the workflow, the typesetting workflow, and the workflow of publishing, and not the lack of AI. 

What are the first concrete steps for integrating AI into production processes?

Most of the checks for accessibility can be performed by humans or by machines without the use of AI, but for sure, AI can help to speed up some time-consuming tasks. For example with generating alt text. If there is a starting point from a generative AI, we can produce alt text quickly, because we have alt text to start with, and then editing it and send it. 

The advice is to try to map the workflow, identifying the tasks that require more time for humans to do, identify which tools can help in accelerating this task, and leaving always the human the last check and the decisions. Many professional tools in generative AI are not automatic. For example, the tools that are available for developers for coding, they process the code, and then they always ask the developer if the code that they produce, is okay. So it is more a collaboration between AI and tools.

How far can AI go in detecting and fixing accessibility issues in existing publications?

Gregorio tells an anecdote to describe the answer. He attended a demo of a system that was designed to automatically make EPUBs accessible, starting from non-accessible EPUBs. The tool was able use a non-accessible EPUB2 as input, and provide as an output an accessible EPUB 3. It was an iterative process, in which a generative AI agent fixed the EPUB. The AI agent was trained to fix the accessibility of the EPUB. Then, when the generative AI agent finished the work, the EPUB was automatically run through AceByDaisy and EPUB Check. These are free and automatic tools for checking accessibility of the EPUB, until it reached zero errors. This approach was however based on two fundamental errors. First: passing it through AceByDaisy and EPUB check, we know that it's not enough to claim that an EPUB is accessible. We know that these two tools are available to check about 30% of issues in accessibility, and not 100%. Second: based on a test file, Gregorio realized that the fixes that the AI was making were not about accessibility. They were about fixing AceByDaisy errors, which is a different thing.

So, automatic remediation tools are not available at the moment, and that may require some years.

Do you think there's a way that AI can be trained by experts to do quality assurance checks correctly and reduce manual testing? 

Yes, for sure. But the fact is that generative AI tools are mostly trained on what is available on the web, and we know that a lot of information available on the web about accessibility is incorrect or not updated.

Use AI as a junior-colleague approach, meaning reviewing the answers, never trust it is correct. Fondazione LIA uses some internal tools that use automations and AI to accelerate the quality control of accessible ebooks. But these are always the starting point in order to present it to an experienced colleague. In many cases, AI, again, is not necessary. Simple programming is sufficient. Maybe with AI, we can identify 50% of the accessibility issues, we don't have the confidence on the result.

When you check quality assurance, there are three levels that are interesting for everyone.

1. The first one is the deterministic approach. This means the old-way of programming. For example, Ace by Daisy and EPUB Check are two tools that are deterministic tools. If they file an issue, it's an accessibility issue for sure. If they don't, it doesn't mean that the file is accessible. But if they file it, it's cause they are 100% sure that that is an issue. The deterministic approach is a good approach to start.
2. Then pass it through a AI press scanning, to check all the things that look like errors, but the AI is not sure if it's an error or not. Or maybe the AI is sure, but we don't trust it. And use this first scan to present the operator meaningful issues that may be present in the content.
3. The last step is always the human check; go through the whole content to check a third time, because AI sometimes has hallucinations, meaning bugs, and do not identify things.

Are there any other solutions or research projects on using AI within the accessible publishing sphere? 

There are different people researching this, for example, researching the use of generative AI to check if an alt text is proper or not for an image. But there isn’t any of this research in production, so available in a tool, just yet.

When an image has been generated using AI, do publishers need to mention that? Should you mention in the Books Color Fund that you used artificial intelligence to help draft the description of an illustration which you later revised?

There is a lot of discussion about it. There are metadata that can say that an alternative description is generated by machines. Also, there are metadata for telling that an audiobook is produced/narrated by AI.

Is AI capable of accurately identifying words that are in different languages, and giving them the appropriate language attributes?

Yes that is possible. At least to identify issues with the language tagging, maybe not fixing them. Is a generative AI capable of doing that? Maybe not. This is really a field where generative AI is not good at. A more machine learning approach is a better one. Or maybe also a deterministic approach. The language attribute is important for people using text-to-speech and the correct voice will speak the text. This is really critical for accessibility.

Is it possible to train AI to reliably extract and generate accessibility metadata from publications? 

You do not really need AI for the extraction of accessibility metadata, but with just some programmatic approach it can be achieved. It's the publisher’s responsibility to create the accessibility metadata, and not any other operator in the supply chain. That means that the publisher should be familiar with how the e-book was created and the content of the e-book. The publisher should know in advance an e-book has alt text or not, because it's a production decision That means we really need a programmatic approach to extract metadata. Having incorrect accessibility metadata is worse than non-metadata, because it creates false expectations for the user, so never publish unverified automated generate metadata.