Experiment Optimization is a powerful analysis tool that helps you understand and optimize your experiments. It's designed to solve complex, multi-dimensional problems where an outcome is affected by many interacting variables.

For example, imagine you want to maximize the protein yield from a bioreactor. The final yield is a function of many factors: pH, temperature, available glucose, dissolved O₂, etc. These variables can have complex relationships with each other and the final output. Some effects may be linear (more glucose leads to more product), while others are non-linear (trace metals may be beneficial up to a point and toxic beyond that). Manually finding the best combination of conditions is difficult and time-consuming.

This tool automates that process by using machine learning to identify the optimal conditions, helping you minimize experimental cycles and costs.

In order to get access to this feature while it is in beta, you must:

• Have a license to: either Advanced Analysis or Bioprocess
• Enable Insights Labs on your tenant 

How does experiment optimization work? 

When you run an Experiment Optimization analysis, Benchling performs a sophisticated, automated workflow:

1. Model Training: The system trains a “tournament” of machine learning models on your data. It works to create a function that, given your set of input parameters, can accurately predict your target outcome (“Target variable”) best. The model output provides the model's overall accuracy (R² value), shows the relative importance of each input variable, and includes features that allow you to predict outcomes based on new conditions. Even though this phase finishes first, most scientists who are not familiar with the specifics of predictive modeling will want to see Recommendations first.

2. Recommendations: Using the best performing model, Bayesian optimization produces batched recommendations using the uncertainty from model fitting.  Specifically, the search space is tested for a collection of recommendations that are most likely to improve the variable of interest.

Prepare your data for success

The quality of your input data directly determines the success of the analysis. Before you begin, review your dataset with these best practices in mind. You can use Analysis to make any of the tweaks below.

• Data Shape Matters: For the best results, your data table should be a "tall rectangle", meaning you have (ideally many) more rows of data (observations) than columns of variables. If the effects you're measuring are weak, you will need more rows for the model to find a meaningful signal.

• Check for Data Leaks: Ensure that none of your input columns are a direct calculation or proxy for your target outcome. For example, if your $Y$ variable is Total Protein (g) and you include an input column for Tank Volume (L), the model might find the obvious (but unhelpful) correlation that larger tanks produce more total protein. Normalize your data to avoid this (e.g., use Protein Concentration (g/L)).

• Remove Redundant Columns: If two columns represent overlapping concepts, it's best to remove one. For instance, if your system measures both %O₂ and %CO₂ and they must sum to 100%, including both columns provides the same information twice.

Use experiment optimization

1. Create a New Analysis

   Begin by creating a new Analysis and uploading or selecting your dataset.

2. Access Experiment Optimization Settings

   In the Analysis table sidebar, click “New View” and select Experiment Optimization, under AI & Machine Learning.

3. Set Configuration Options

   • Target column: This is the most important setting. It's the primary outcome or result you want to optimize. The model will work to find the experimental conditions that maximize the value in this column.

     • Example: To find settings that improve product yield, you would select the Final product concentration (mg/L) column.

   • Grouping column (Optional): Use this field if your dataset contains replicates or related runs. You should select the column that identifies which rows belong to the same group (e.g., a Replicate ID, Batch, or Tank column). This helps the model understand the underlying structure of your experiment.

   • Number of recommendations: Enter the number of optimized condition sets you want the model to generate. The final report will suggest this many different combinations of your input parameters that are predicted to give the best results for your target.

     • Example: Entering 5 will provide you with the top five recommended sets of conditions to test in your next experiment.
       
       

4. Run the Optimization

   After configuring the settings, submit the form. Benchling will process the data, and an Experiment Optimization section will appear in your Analysis once the results are ready. This can take up to an hour.

Experiment optimization sections

Recommendations - configure design space

• The boundaries of what the optimization will explore as possible conditions to recommend are set here.

• These values are set initially based on the range in the input dataset used for the experiment optimization run, and can be manually changed. To regenerate the recommendations, click the “Generate recommendations” button.

• You may want to change the Step size to “zoom in” on the most important variables.

Recommendations

Each row in the table is a candidate, or suggested set of experiment parameters.

Performance of considered recommendations

This chart shows how well the Recommended conditions are expected to perform based on the target metric, how they compare with conditions that were already tested (“Observed”), and other candidates that were neither tested nor recommended (“Predicted”).

This chart helps you see the spread of predicted performance, and how the recommendations balance explore and exploit to give a diverse set of recommendations.

Explore recommendations - 2D surfaces

The 2D surface charts allow you to select two parameters and compare how their combined effects are estimated to affect performance, and how confident the model is in that estimate.

Performance: Model’s predicted outcomes on the combination of the two variables. Darker areas mean higher predicted performance.

Uncertainty: Model’s confidence in its predictions in each area. Darker areas mean more uncertainty or less confidence.

How to use

• Start by looking at the two most important features, as shown in the bar chart on the “Model details” page.

• Explore parameter interactions: Change parameters for the x and y axis to see how different combinations affect results. 

• Spot promising areas: Use the performance plot to find regions likely to give strong results.

Balance risk vs confidence. Areas with low uncertainty are safer bets, while areas with higher uncertainty may be riskier, but could lead to unexpectedly good results.

Explore recommendations - 1D slices

The 1D slice chart lets you see how changing any important variable affects the predicted outcome while holding the other variables constant.

1. Recommendation: Pick which recommendation from the Recommendations table above to inspect

Parameter: Pick which parameter to vary; any parameter not selected will be held constant (according to the recommendation picked above) and shown as “Additional parameters”. (Start with your most important parameter from the “Model details” page.)

Predicted: Model’s expected performance at different values of the chosen parameter.

Recommended: Specific value the system suggestions for this recommendation.

If there are observed data points, they will also be shown.

How to use

• Understand sensitivity: Observe how much performance changes when you vary a particular parameter, while holding other values constant.

• Validation: Evaluate whether the recommendations look to be in a good region.

Model details

This section explains the model details page and how to interpret the results.

The first table gives basic information about the inputs and outputs of the model that were set up at the creation of this run of Experiment Optimization.

The Parameterssection shows all the columns that went into training. Some will be removed because they are not found to contribute to the model’s performance. They will be shown in a separate section immediately beneath it labeled “Unused Parameters.”

Training metrics

Training metrics show how well the model fits the data. If the data is large enough, the original data is split into a Training set, and a Test set, so that the model is “tested” for performance on data that it wasn’t exposed to during training.

How to interpret

The R² score explains how much of the variation of the output is explained by the inputs. Do not confuse this with P values. Even models with very low R² can still be statistically significant. The closer R² is to 1, the better the inputs explain the output data.

Cross-validation is used to make sure the model is not just memorizing (overfitting), but learning useful patterns. The training dataset is split into chunks. Each round, a set of chunks are used for training, and is tested on the rest. This is repeated using different held-out chunks each time. The model from the cross-validation phase has only been trained using the training data, not the test data. The model’s built-in settings (hyperparameters) are tuned by cross-validation to avoid overfitting.

The “final” measurement is for a model that is trained after adding the test data back in (so no information is left unused) but by a model that is prevented from overfitting via the cross-validation phase.

Predicted vs actual graph

This chart is a visual version of the Training Metrics table. It shows how well the predicted performance outcomes are to the actual performance outcomes for the test data (red) and training data (blue). The closer these plots are to a diagonal line, the more accurate the model is, and the higher the R² will be.

Caution

• R² above 0.9: The model may be overfitting. R² this high is not likely to be genuine.

• R² <= 0: It is possible though unlikely that the model succeeds with low or even negative R² outcomes. This is because R² is a poor metric when there are very few data points.

Parameter importance & contribution

These two views work together to explain which parameters matter most and how the values of those variables influence predictions.

• Parameter importance: how much each parameter contributes overall

  • The taller the bar, the stronger the influence on the outcome.

• Parameter contribution: how each data point was affected by that parameter. There is one dot per row in the training data. Sometimes many points get stacked.

  • Positive values → that value of that variable causes an increase in the predicted value.

  • Negative values → that value of the variable causes a decrease in the predicted value.

  • Points hover around zero → little effect

  • Sometimes you can see an interesting trend, and conclude things like “increasing temperature is associated with an increase in my measured outcome”.

Parameter contribution

• Scatter plot shows how each data point was affected

  • If a parameter had small bar in the Importance chart, it would be expected that the Contribution y-axis would hover around zero.

  • If a parameter had a large bar, it would be expected that the Contribution would show lots of variation and points would be spread out from zero. Each point would show a nudge up or down in the prediction.

     

Model class comparison

The modeling process runs a tournament of machine learning models. We shine a little light into this via the Model Class Comparison chart.

 

• X-axis: Different regression algorithm classes tested

• Y-axis: Average performance score during cross-validation (higher = better)

How to interpret

• Each mark shows how consistently the model performed across multiple runs.

• This gives the user a sense of the model classes used, and which tended to fit their data best.

Frequently Asked Questions

Q: Why did my analysis fail?
A: The most common reason for failure is that no model achieved a predictive power (R² value) greater than 0.10. This typically means there isn't a strong enough signal in your input data to predict the outcome. Try cleaning your data further using the best practices above or adding more experimental data (rows) to your table.

Q: How much data do I need?
A: There's no magic number, but the "tall rectangle" rule is a good guide: you need more rows (data points) than columns (variables). The weaker the relationships you are trying to model, the more data you will need.

Q: Is the model a simple regression?
A: No, the tool uses a more complex combination of machine learning and Bayesian inference approach. It creates a predictive function based on your inputs and outputs, but it does not provide a simple linear equation.