If you already have a SIP phone system, you can give an AI voice agent its own extension. A caller dials the extension, speaks normally, and hears the OpenAI Realtime model’s reply on the same call.

I’m affiliated with VaxVoIP, and this walkthrough uses our VaxVoIP SIP Call OpenAI SDK for Windows.

What the gateway does

SIP phone or PBX ↔ C# gateway ↔ OpenAI Realtime

SIP sets up and ends the phone call, and RTP carries the audio. The SDK handles the SIP stack, bridges audio in both directions, and connects to OpenAI Realtime. You can build the gateway without writing WebRTC code or implementing a SIP stack.

Your C# application decides which calls to accept, what instructions to give the voice agent, and which application functions the agent can use.

Try it with the C# sample

Watch the C# SIP-to-OpenAI gateway demo on YouTube

  1. Download the C# sample project, extract it on Windows, and open the project in Visual Studio.
  2. Configure your OpenAI API key in the product-order demo. Load the key from an environment variable or a secret store to keep it out of source control.
  3. Run the sample, register a SIP softphone with the gateway, and dial 101. Ask the product-order agent whether it supports laptops and listen for the spoken reply.

After preparing the prompt, model, and tools, the sample’s incoming-call handler accepts the call with one SDK call.

base.AcceptCallSession(30, KeyOpenAI, ModelPrompt, Model, "coral", 1.0);

The sample also shows how to connect the voice agent to your application data. When the caller asks about a product, the model can request a lookup. The C# application checks the product and returns the result so the model can answer the caller.

Once the softphone test works, route a test extension from your PBX to the gateway. For an internal help desk, you could change the prompt and replace the product lookup with a function that retrieves support hours or checks ticket status.

VaxVoIP SIP Agent OpenAI SDK and documentation

What you can build

  1. AI receptionists that greet callers, find out what they need, and route them to the right person or department.
  2. Customer support agents that answer common questions, check order or ticket status, and hand off to a human when needed.
  3. Telemarketing agents that call opted-in leads, ask qualifying questions, and book follow-up calls.
  4. Smart AI autodialers that make reminder calls, return callbacks, run post-service surveys, and log call outcomes.
  5. Smart IVRs that let callers explain what they need in their own words and route them to the right queue or self-service option.
  6. Appointment booking agents that check availability, book or reschedule appointments, and confirm the details over the phone.
  7. Order-taking agents that check product availability, collect order details, and read them back for confirmation.
  8. Multilingual voice agents that answer questions in the caller’s preferred language and transfer calls when human help is needed.
  9. Internal help desk agents that check ticket status or gather the information needed to open a new ticket.