Using AI for writing study reports is a powerful, but somewhat fraught, tool. The AI often does not have full information about the science you are conducting, and often biases towards producing output that is the "average of the training set", which is not always of scientific quality.
Benchling's AI report generation feature does a set of base prompts that make the AI produce output that we believe is more robust and useful for scientific reporting purposes. We won't give all the details about these base prompts (and we are changing them frequently), but overall our guidance includes telling the AI it should behave and write like a scientist, it should cite its sources, and it should not attempt to do numerical calculations.
We have found that users using the report generation capability often ask for specific output from the AI and are relatively precise about what they need. Some examples:
- [[Generate a scientific report. Each section should contain between 200 and 500 words. DO NOT give descriptive statements or claims. DO NOT use adjectives or adverbs. Provide factual summaries.]]
- [[Provide factual summaries. The following sections should be included in the report: 1) Introduction - An introduction for a study report, give source citations. 2) Assays - A description of the assays conducted in all included notebook entries. 3) Conclusion - A short conclusion that summarizes all work that was performed.]]
In addition, we have found the following recommendations to be helpful for certain scientific use cases:
- Use a single prompt rather than a series of prompts in the template. The AI is already pretty good at writing up sections, so you don't necessarily need to section the template directly.
- Give the AI context into the overall science your company is doing. Sometimes this is done by creating a single notebook entry with information about the company overall, your particular scientific approach (if that is therapeutic area or modality specific), and the goals for the program or study being examined. Sometimes this context is not in the notebook entries.
- Similarly, we see some notebook entries are exclusively tables and pictures. The AI is unlikely to be very good at summarization in these cases, as it best understands written text and/or bullets. We do provide the AI the table information, but it doesn't have a full understanding of all schema metadata fields or associated data, unless that information is already in the entry.