Let AI choose which function to call

Let AI choose which function to call

The chat completion endpoint endpoint of the OpenAI API offers an astonishing feature. It is possible to let the AI choose which function to call based on the textual input and a given list of python function signatures. In this blog post I will take a look how to use this API and how explore a way how to use it in an application.

The Setup

I created a file with some very basic python functions, one of the being a method to get tbe current balance of a bank account. Other functions in the same file are not related to banking at all but are just there to test the AI’s ability to choose the right function. Here is a list of the functions signatures:

def get_balance(account_number: str) -> float:

...

def get_account_numbers() -> list:

...

def get_account_number(name: str) -> str:

...

def measure_scale_balance(balance: float, scale: float) -> float:

The complete code of this example can be found here . When using the chat completion endpoint for function calling, the request body must contain the the signature of the functions in a specific format (see here ) for details. Instead of manually creating the list of functions from the python file, I wrote a functions which does that given some basic constraints like using only specific primitive types:

import inspect

def get_function_as_dict(function_name: str):
    function = globals()[function_name]
    function_signature = inspect.signature(function)
    function_signature_as_dict = {"name": function_name, "description": function.__doc__,
                                  "parameters": {"type": "object", "properties": {}, "required": []}}
    for parameter in function_signature.parameters:
        parameter_type = function_signature.parameters[parameter].annotation.__name__
        type_mapping = {"str": "string", "float": "number", "int": "integer", "list": "array"}
        parameter_type = type_mapping[parameter_type]

        function_signature_as_dict["parameters"]["properties"][parameter] = {
            "type": parameter_type }
        function_signature_as_dict["parameters"]["required"].append(parameter)

    return function_signature_as_dict

The functions uses pythons inspect module to get the function signature and convert these to a dictionary which can be used in the request body to the chat completion endpoint (The complete code can be found here ). Here is an example of the json representation of the function get_balance generated with this function:

{
  "name": "get_balance",
  "description": "\n    Return the balance of the account identified by the account number\n    :param account_number: The account number\n    :return: The balance of the account number\n    ",
  "parameters": {
      "type": "object",
      "properties": {
          "account_number": {
              "type": "string"
          }
      },
      "required": [
          "account_number"
      ]
  }
}

Having some more or less meaningful functions to choose from and some basic tooling, it’s time to put it all together.

Using the chat completion endpoint to choose a function

Invoking the chat completion endpoint is straight forward. The request body must contain the prompt and the list of functions to choose from.

messages = [{"role": "user", "content": prompt}, {"role": "system", "content": "Banking account program"}]

functions = [get_function_as_dict(function_name) for function_name in globals() if callable(globals()[function_name])]

response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0613",
    messages=messages,
    functions=functions)["choices"][0]["message"]

In the first line I assign a specific role to the model. That makes responses more domain specific and predictable. The second line creates the list of functions from the globals dictionary. Note that this code creates function signatures for all functions in the file not just the ones related to banking. Also note that I only include objects which are callable since globals also contains other objects.

If all went according to plan, the response should contain get_balance as the function to call. The following code gets the function name and invokes the function using the parameter values in the response as arguments:

if response.get("function_call"):
    function_name = response["function_call"]["name"]

    all_functions = {function.__name__: function for function in globals().values() if callable(function)}
    function_to_call = all_functions[function_name]
    function_args = json.loads(response["function_call"]["arguments"])

    return function_to_call(**function_args)

First I check whether a function was found at all and if so, I extract the name of the function to call. Then I create a dictionary of all functions in the file and get the function to call from that dictionary using its name. The arguments returned are in json format as a string. I convert them back to a dictionary and invoke the function using the ** operator. That assumes that the arguments returned are in the same order. The complete source code for this function which I called get_balance_with_prompt can be found here .

Works for any language

For testing function calling I implemented a test case which can be found here . The test case invokes my function passing a prompt as input. For example:

response = get_balance_with_prompt("My account number is A-1234 and I want to know my balance")
self.assertEqual(100, response)

The test case passes indicating that the right method was called. Now here it gets interesting. I can change the language of the prompt to other languages and the code still works:

French: Mon numéro de compte est A-1234 et je veux connaître mon solde
Hebrew: מספר החשבון שלי הוא A-1234 ואני רוצה לדעת את היתרה שלי

Talking about the need to internationalize applications…

Conclusion

The ability to choose a function to call from textual input in almost any language is quiet impressive. Thinking about a chatbot interaction with a user, it could be possible to automatically invoke a function based on the user input. No additional code needed. Calling arbitrary code on behalf of a user obviously has mayor security implications. In addition predictability of the response is important in order to get equal results for the same input. Fine tuning might be an affective way to achieve this.

Back to Home