Feat: Override Valves Per-model

by ADMIN 32 views

Introduction

In the current implementation of Open-webui, function configuration has two modalities: global system-level valves and user-specific valves. This setup makes it challenging to create filters customized per model, as users need to copy-paste function code, give it a new name, and set system-level valves to a different configuration. This article proposes a solution to allow system-level valves to be overridden on a per-model level, enhancing user experience and usability.

Problem Description

The current function configuration system has two primary modalities:

  • Global System-Level Valves: These valves are applicable to the entire Open-webui instance and are not specific to individual users or models.
  • User-Specific Valves: These valves are tailored to an individual user's preferences and are not shared across users.

To create a filter customized per model, users must:

  1. Copy-Paste Function Code: Duplicate the underlying function code to create a new copy.
  2. Give it a New Name: Assign a unique name to the new function copy.
  3. Set System-Level Valves: Configure system-level valves to a different setting.
  4. Instruct Users: Inform users which function copy to use for a specific configuration.

This process is cumbersome and makes it difficult for a large number of users to maintain and customize their own model configurations with various valves.

Desired Solution

To address this issue, Open-webui should allow system-level valves to be overridden on a per-model level. This would require:

  1. User Interface: Adding a user interface to enumerate and configure valves on the model edit page.
  2. Model Configuration Storage: A place to store the overridden valves in the model configuration.
  3. Back-end Logic: Exposing the proper valve to the function on use.

Alternatives Considered

Before proposing the desired solution, several alternatives were considered:

  • Create Multiple Copies of the Same Function: This approach has several drawbacks, including:
    • Sharp Edges: Confusing user interactions and duplicated code.
    • Code Duplication: Repeated code makes maintenance and updates challenging.
  • Add Custom Metadata to Models: This approach involves adding custom metadata to models that can be accessed in function code to manually override system-level valves. While this is a current hack, it would be more usable if it duplicated the DX (for function authors) and UX (just add the valve config UI to the edit model page) of valves.
  • Add a New "Model Valve" Data Type to Functions: This approach would not be backwards compatible with current valve code and is unclear about the benefits of explicitly exposing both instance-wide and per-model valves as different configuration options.

Additional Context

The author is willing to work on a PR to implement this solution, which would enhance the usability and user experience of Open-webui.

Benefits of the Proposed Solution

The proposed solution would provide several benefits, including:

  • Improved User Experience: Users would no longer need to copy-paste function code, giving them a more streamlined and efficient experience.
  • Enhanced Usability: The addition of a user interface to enumerate and configure valves on the model edit page would make it easier for users to customize their model configurations.
  • Simplified Maintenance: With the ability to override system-level valves on a per-model level, maintenance and updates would become more manageable.

Conclusion

Frequently Asked Questions

Q: What is the current issue with function configuration in Open-webui? A: The current function configuration system has two modalities: global system-level valves and user-specific valves. This setup makes it challenging to create filters customized per model, as users need to copy-paste function code, give it a new name, and set system-level valves to a different configuration.

Q: What is the proposed solution to address this issue? A: The proposed solution is to allow system-level valves to be overridden on a per-model level. This would require adding a user interface to enumerate and configure valves on the model edit page, storing the overridden valves in the model configuration, and exposing the proper valve to the function on use.

Q: What are the benefits of the proposed solution? A: The proposed solution would provide several benefits, including:

  • Improved User Experience: Users would no longer need to copy-paste function code, giving them a more streamlined and efficient experience.
  • Enhanced Usability: The addition of a user interface to enumerate and configure valves on the model edit page would make it easier for users to customize their model configurations.
  • Simplified Maintenance: With the ability to override system-level valves on a per-model level, maintenance and updates would become more manageable.

Q: What alternatives were considered before proposing the solution? A: Several alternatives were considered, including:

  • Create Multiple Copies of the Same Function: This approach has several drawbacks, including confusing user interactions and duplicated code.
  • Add Custom Metadata to Models: This approach involves adding custom metadata to models that can be accessed in function code to manually override system-level valves.
  • Add a New "Model Valve" Data Type to Functions: This approach would not be backwards compatible with current valve code and is unclear about the benefits of explicitly exposing both instance-wide and per-model valves as different configuration options.

Q: Who is willing to work on a PR to implement this solution? A: The author of this proposal is willing to work on a PR to implement this solution, which would enhance the usability and user experience of Open-webui.

Q: What is the expected outcome of implementing this solution? A: The expected outcome of implementing this solution is to provide a more efficient and streamlined experience for users, making it easier for them to customize their model configurations and simplifying maintenance and updates.

Q: How will this solution impact the Open-webui community? A: This solution will positively impact the Open-webui community by providing a more user-friendly and efficient experience, making it easier for users to customize their model configurations and simplifying maintenance and updates.

Q: What is the next step in implementing this solution? A: The next step in implementing this solution is to work on a PR to add the necessary features and functionality to Open-webui, including the user interface to enumerate and configure valves on the model edit page, storing the overridden valves in the model configuration, and exposing the proper valve to the function on use.

Q: How can the Open-webui community get involved in implementing this solution? A: The Open-webui community can get involved in implementing this solution by contributing to the PR, providing feedback and suggestions, and helping to test and refine the new features and functionality.