Interface SourceOutput<T>

    • Method Detail

      • collect

        void collect​(T record)
        Emit a record without a timestamp.

        Use this method if the source system does not have a notion of records with timestamps.

        The events later pass through a TimestampAssigner, which attaches a timestamp to the event based on the event's contents. For example a file source with JSON records would not have a generic timestamp from the file reading and JSON parsing process, and thus use this method to produce initially a record without a timestamp. The TimestampAssigner in the next step would be used to extract timestamp from a field of the JSON object.

        Parameters:
        record - the record to emit.
      • collect

        void collect​(T record,
                     long timestamp)
        Emit a record with a timestamp.

        Use this method if the source system has timestamps attached to records. Typical examples would be Logs, PubSubs, or Message Queues, like Kafka or Kinesis, which store a timestamp with each event.

        The events typically still pass through a TimestampAssigner, which may decide to either use this source-provided timestamp, or replace it with a timestamp stored within the event (for example if the event was a JSON object one could configure aTimestampAssigner that extracts one of the object's fields and uses that as a timestamp).

        Parameters:
        record - the record to emit.
        timestamp - the timestamp of the record.