> ## Documentation Index
> Fetch the complete documentation index at: https://private-7c7dfe99-postgresql-tls-support.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Discover ClickPipe source schema

> **This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future. <br /><br /> Infers the schema (field names and ClickHouse data types) of a streaming ClickPipe source without creating a pipe. Supported for Kafka and Kinesis sources.

<span data-endpoint-badge="Beta"><Badge color="blue">Beta</Badge></span>

**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future. <br /><br /> Infers the schema (field names and ClickHouse data types) of a streaming ClickPipe source without creating a pipe. Supported for Kafka and Kinesis sources.


## OpenAPI

````yaml _specs/cloud-openapi.json POST /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/schemaDiscovery
openapi: 3.1.2
info:
  title: OpenAPI spec for ClickHouse Cloud
  version: '1.0'
  contact:
    name: ClickHouse Support
    url: >-
      https://clickhouse.com/docs/en/cloud/manage/openapi?referrer=openapi-910487
    email: support@clickhouse.com
servers:
  - url: https://api.clickhouse.cloud
security:
  - basicAuth: []
tags:
  - name: Organization
  - name: User management
  - name: Billing
  - name: Role Management
  - name: Service
  - name: Backup
  - name: OpenAPI
  - name: Prometheus
  - name: ClickPipes
  - name: ClickStack
  - name: Postgres
paths:
  /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/schemaDiscovery:
    post:
      tags:
        - ClickPipes
      summary: Discover ClickPipe source schema
      description: >-
        **This endpoint is in beta.** API contract is stable, and no breaking
        changes are expected in the future. <br /><br /> Infers the schema
        (field names and ClickHouse data types) of a streaming ClickPipe source
        without creating a pipe. Supported for Kafka and Kinesis sources.
      operationId: clickPipeSchemaDiscovery
      parameters:
        - in: path
          name: organizationId
          description: ID of the organization that owns the service.
          required: true
          schema:
            type: string
            format: uuid
        - in: path
          name: serviceId
          description: ID of the service to run schema discovery against.
          required: true
          schema:
            type: string
            format: uuid
      requestBody:
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/ClickPipeSchemaDiscoveryRequest'
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: number
                    description: HTTP status code.
                    example: 200
                  requestId:
                    type: string
                    description: Unique id assigned to every request. UUIDv4
                    format: uuid
                  result:
                    $ref: '#/components/schemas/ClickPipeSchemaDiscoveryResponse'
        '400':
          description: >-
            The request cannot be processed due to a client error. Please verify
            your request parameters and try again.
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: number
                    description: HTTP status code.
                    example: 400
                  error:
                    type: string
                    description: Detailed error description.
                  requestId:
                    type: string
                    description: Unique id assigned to every request. UUIDv4
                    format: uuid
        '500':
          description: >-
            An internal server error has occurred. If this issue persists,
            please contact ClickHouse Cloud support for assistance.
          content:
            application/json:
              schema:
                type: object
                properties:
                  status:
                    type: integer
                    description: HTTP status code.
                    example: 500
                  error:
                    type: string
                    description: Detailed error description.
                  requestId:
                    type: string
                    description: Unique id assigned to every request. UUIDv4
                    format: uuid
components:
  schemas:
    ClickPipeSchemaDiscoveryRequest:
      properties:
        source:
          $ref: '#/components/schemas/ClickPipeSchemaDiscoverySource'
    ClickPipeSchemaDiscoveryResponse:
      properties:
        fields:
          type: array
          description: Inferred schema fields with their ClickHouse data types.
          items:
            $ref: '#/components/schemas/ClickPipeSchemaDiscoveryField'
    ClickPipeSchemaDiscoverySource:
      properties:
        kafka:
          oneOf:
            - $ref: '#/components/schemas/ClickPipePostKafkaSource'
            - type: 'null'
        kinesis:
          oneOf:
            - $ref: '#/components/schemas/ClickPipePostKinesisSource'
            - type: 'null'
    ClickPipeSchemaDiscoveryField:
      properties:
        name:
          description: Name of the inferred field.
          type: string
          example: user_id
        type:
          description: Inferred ClickHouse data type of the field.
          type: string
          example: Int64
        optional:
          description: Whether the field is optional (nullable) in the source.
          type:
            - boolean
            - 'null'
    ClickPipePostKafkaSource:
      properties:
        type:
          description: Type of the Kafka source.
          type: string
          enum:
            - kafka
            - redpanda
            - msk
            - gcmk
            - confluent
            - warpstream
            - azureeventhub
            - dokafka
        format:
          description: Format of the Kafka source.
          type: string
          enum:
            - JSONEachRow
            - Avro
            - AvroConfluent
            - Protobuf
        brokers:
          description: Brokers of the Kafka source.
          type: string
        topics:
          description: Topics of the Kafka source.
          type: string
        consumerGroup:
          description: >-
            Consumer group of the Kafka source. If not provided
            "clickpipes-<<ID>>" will be used.
          type:
            - string
            - 'null'
          example: my-clickpipe-consumer-group
        authentication:
          description: >-
            Authentication method of the Kafka source. Supported authentication
            methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS,
            msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN,
            MUTUAL_TLS, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN,
            azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512,
            MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS
          type: string
          enum:
            - PLAIN
            - SCRAM-SHA-256
            - SCRAM-SHA-512
            - IAM_ROLE
            - IAM_USER
            - MUTUAL_TLS
        iamRole:
          description: >-
            IAM role for the Kafka source. Use with IAM role authentication.
            Read more in ClickPipes documentation:
            https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam
          type:
            - string
            - 'null'
          example: arn:aws:iam::123456789012:role/MyRole
        offset:
          oneOf:
            - $ref: '#/components/schemas/ClickPipeKafkaOffset'
            - type: 'null'
        schemaRegistry:
          oneOf:
            - $ref: '#/components/schemas/ClickPipeMutateKafkaSchemaRegistry'
            - type: 'null'
        caCertificate:
          description: PEM encoded CA certificates to validate the broker's certificate.
          type:
            - string
            - 'null'
        reversePrivateEndpointIds:
          type: array
          description: >-
            Reverse private endpoint UUIDs used for a secure private connection
            to the Kafka source.
          items:
            type: string
        exactlyOnce:
          description: >-
            Enable exactly-once delivery. Guarantees every Kafka record is
            inserted exactly once across restarts and rebalances. Can only be
            set at pipe creation.
          type:
            - boolean
            - 'null'
        credentials:
          description: >-
            Credentials for Kafka source. Choose one that is supported by the
            authentication method.
          oneOf:
            - $ref: '#/components/schemas/PLAIN'
            - $ref: '#/components/schemas/MskIamUser'
            - $ref: '#/components/schemas/AzureEventHub'
            - $ref: '#/components/schemas/MutualTLS'
    ClickPipePostKinesisSource:
      properties:
        format:
          description: Format of the Kinesis stream.
          type: string
          enum:
            - JSONEachRow
            - Avro
            - AvroConfluent
        streamName:
          description: Name of the Kinesis stream.
          type: string
          example: my-stream
        region:
          description: AWS region of the Kinesis stream.
          type: string
          example: us-east-1
        useEnhancedFanOut:
          description: Use enhanced fan-out for the Kinesis stream.
          type:
            - boolean
            - 'null'
        iteratorType:
          description: >-
            Type of iterator to use when reading from the Kinesis stream. If
            AT_TIMESTAMP is used, the timestamp field must be provided.
          type: string
          enum:
            - TRIM_HORIZON
            - LATEST
            - AT_TIMESTAMP
        timestamp:
          description: >-
            UNIX timestamp to start reading from the Kinesis stream. Required if
            iteratorType is AT_TIMESTAMP.
          type:
            - integer
            - 'null'
          example: 1615766400
        authentication:
          description: Authentication method to use with the Kinesis stream.
          type: string
          enum:
            - IAM_ROLE
            - IAM_USER
        iamRole:
          description: IAM role to use for authentication. Required if IAM_ROLE is used.
          type:
            - string
            - 'null'
          example: arn:aws:iam::123456789012:role/MyRole
        accessKey:
          oneOf:
            - $ref: '#/components/schemas/MskIamUser'
            - type: 'null'
    ClickPipeKafkaOffset:
      properties:
        strategy:
          description: Offset strategy.
          type: string
          enum:
            - from_beginning
            - from_latest
            - from_timestamp
        timestamp:
          description: >-
            A minute precision UTC timestamp to start from. Required for
            "from_timestamp" strategy.
          type:
            - string
            - 'null'
          example: 2021-01-01T00:00
    ClickPipeMutateKafkaSchemaRegistry:
      properties:
        url:
          description: Schema URL. HTTPS required.
          type: string
          example: https://psrc-aa00.us-east-2.aws.confluent.cloud/schemas/ids/100004
        authentication:
          description: Authentication type of the schema registry.
          type: string
          enum:
            - PLAIN
        caCertificate:
          description: >-
            PEM encoded CA certificates to validate the schema registry's
            certificate.
          type:
            - string
            - 'null'
        credentials:
          $ref: '#/components/schemas/ClickPipeKafkaSchemaRegistryCredentials'
    PLAIN:
      properties:
        username:
          description: Database username.
          type: string
          example: postgres_user
        password:
          description: Database password.
          type: string
          format: password
          example: your_secure_password
    MskIamUser:
      properties:
        accessKeyId:
          description: IAM access key ID.
          type: string
        secretKey:
          description: IAM secret key.
          type: string
    AzureEventHub:
      properties:
        connectionString:
          description: Connection string for Azure EventHub source.
          type: string
    MutualTLS:
      properties:
        certificate:
          description: PEM encoded client certificate for mTLS authentication.
          type: string
        privateKey:
          description: PEM encoded client private key for mTLS authentication.
          type: string
          format: password
    ClickPipeKafkaSchemaRegistryCredentials:
      properties:
        username:
          description: Username for the schema registry.
          type: string
        password:
          description: Password for the schema registry.
          type: string
  securitySchemes:
    basicAuth:
      type: http
      scheme: basic
      description: >-
        Use key ID and key secret obtained in ClickHouse Cloud console:
        https://clickhouse.com/docs/cloud/manage/openapi

````