com.fasterxml.jackson.databind.util

## Class TokenBuffer

- java.lang.Object
- - com.fasterxml.jackson.core.JsonGenerator
  - - com.fasterxml.jackson.databind.util.TokenBuffer

- All Implemented Interfaces:
    Versioned, Closeable, Flushable, AutoCloseable

      public class TokenBuffer
      extends JsonGenerator

  Utility class used for efficient storage of `JsonToken` sequences, needed for temporary buffering. Space efficient for different sequence lengths (especially so for smaller ones; but not significantly less efficient for larger), highly efficient for linear iteration and appending. Implemented as segmented/chunked linked list of tokens; only modifications are via appends.

  Note that before version 2.0, this class was located in the "core" bundle, not data-binding; but since it was only used by data binding, was moved here to reduce size of core package

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `protected static class` | `TokenBuffer.Parser`
    `protected static class` | `TokenBuffer.Segment` Individual segment of TokenBuffer that can store up to 16 tokens (limited by 4 bits per token type marker requirement).

    - ### Nested classes/interfaces inherited from class com.fasterxml.jackson.core.JsonGenerator

      `JsonGenerator.Feature`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected int` | `_appendAt` Offset within last segment,
    `protected boolean` | `_closed`
    `protected TokenBuffer.Segment` | `_first` First segment, for contents this buffer has
    `protected boolean` | `_forceBigDecimal` Flag set during construction, if use of `BigDecimal` is to be forced on all floating-point values.
    `protected int` | `_generatorFeatures` Bit flag composed of bits that indicate which `JsonGenerator.Feature`s are enabled.
    `protected boolean` | `_hasNativeId` Do we currently have a native type or object id buffered?
    `protected boolean` | `_hasNativeObjectIds`
    `protected boolean` | `_hasNativeTypeIds`
    `protected TokenBuffer.Segment` | `_last` Last segment of this buffer, one that is used for appending more tokens
    `protected boolean` | `_mayHaveNativeIds`
    `protected ObjectCodec` | `_objectCodec` Object codec to use for stream-based object conversion through parser/generator interfaces.
    `protected Object` | `_objectId` If native object ids supported, this is the id for following value (or first token of one) to be written.
    `protected Object` | `_typeId` If native type ids supported, this is the id for following value (or first token of one) to be written.
    `protected JsonWriteContext` | `_writeContext`
    `protected static int` | `DEFAULT_GENERATOR_FEATURES`

    - ### Fields inherited from class com.fasterxml.jackson.core.JsonGenerator

      `_cfgPrettyPrinter`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `TokenBuffer(JsonParser p)`
    `TokenBuffer(JsonParser p, DeserializationContext ctxt)`
    `TokenBuffer(ObjectCodec codec)` Deprecated. since 2.3 preferred variant is one that takes `JsonParser` or additional boolean parameter.
    `TokenBuffer(ObjectCodec codec, boolean hasNativeIds)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected void` | `_append(JsonToken type)`
    `protected void` | `_append(JsonToken type, Object value)`
    `protected void` | `_appendRaw(int rawType, Object value)`
    `protected void` | `_appendValue(JsonToken type)` Similar to `_append(JsonToken)` but also updates context with knowledge that a scalar value was written
    `protected void` | `_appendValue(JsonToken type, Object value)` Similar to `_append(JsonToken,Object)` but also updates context with knowledge that a scalar value was written
    `protected void` | `_reportUnsupportedOperation()`
    `TokenBuffer` | `append(TokenBuffer other)` Helper method that will append contents of given buffer into this buffer.
    `JsonParser` | `asParser()` Method used to create a `JsonParser` that can read contents stored in this buffer.
    `JsonParser` | `asParser(JsonParser src)`
    `JsonParser` | `asParser(ObjectCodec codec)` Method used to create a `JsonParser` that can read contents stored in this buffer.
    `boolean` | `canWriteBinaryNatively()` Since we can efficiently store `byte[]`, yes.
    `boolean` | `canWriteObjectId()`
    `boolean` | `canWriteTypeId()`
    `void` | `close()`
    `void` | `copyCurrentEvent(JsonParser p)`
    `void` | `copyCurrentStructure(JsonParser p)`
    `TokenBuffer` | `deserialize(JsonParser p, DeserializationContext ctxt)` Helper method used by standard deserializer.
    `JsonGenerator` | `disable(JsonGenerator.Feature f)`
    `JsonGenerator` | `enable(JsonGenerator.Feature f)`
    `JsonToken` | `firstToken()`
    `void` | `flush()`
    `TokenBuffer` | `forceUseOfBigDecimal(boolean b)`
    `ObjectCodec` | `getCodec()`
    `int` | `getFeatureMask()`
    `JsonWriteContext` | `getOutputContext()`
    `boolean` | `isClosed()`
    `boolean` | `isEnabled(JsonGenerator.Feature f)`
    `JsonGenerator` | `overrideStdFeatures(int values, int mask)`
    `void` | `serialize(JsonGenerator gen)` Helper method that will write all contents of this buffer using given `JsonGenerator`.
    `JsonGenerator` | `setCodec(ObjectCodec oc)`
    `JsonGenerator` | `setFeatureMask(int mask)` Deprecated.
    `String` | `toString()`
    `JsonGenerator` | `useDefaultPrettyPrinter()`
    `Version` | `version()`
    `void` | `writeBinary(Base64Variant b64variant, byte[] data, int offset, int len)`
    `int` | `writeBinary(Base64Variant b64variant, InputStream data, int dataLength)` Although we could support this method, it does not necessarily make sense: we can not make good use of streaming because buffer must hold all the data.
    `void` | `writeBoolean(boolean state)`
    `void` | `writeEmbeddedObject(Object object)`
    `void` | `writeEndArray()`
    `void` | `writeEndObject()`
    `void` | `writeFieldName(SerializableString name)`
    `void` | `writeFieldName(String name)`
    `void` | `writeNull()`
    `void` | `writeNumber(BigDecimal dec)`
    `void` | `writeNumber(BigInteger v)`
    `void` | `writeNumber(double d)`
    `void` | `writeNumber(float f)`
    `void` | `writeNumber(int i)`
    `void` | `writeNumber(long l)`
    `void` | `writeNumber(short i)`
    `void` | `writeNumber(String encodedValue)`
    `void` | `writeObject(Object value)`
    `void` | `writeObjectId(Object id)`
    `void` | `writeRaw(char c)`
    `void` | `writeRaw(char[] text, int offset, int len)`
    `void` | `writeRaw(SerializableString text)`
    `void` | `writeRaw(String text)`
    `void` | `writeRaw(String text, int offset, int len)`
    `void` | `writeRawUTF8String(byte[] text, int offset, int length)`
    `void` | `writeRawValue(char[] text, int offset, int len)`
    `void` | `writeRawValue(String text)`
    `void` | `writeRawValue(String text, int offset, int len)`
    `void` | `writeStartArray()`
    `void` | `writeStartObject()`
    `void` | `writeStartObject(Object forValue)`
    `void` | `writeString(char[] text, int offset, int len)`
    `void` | `writeString(SerializableString text)`
    `void` | `writeString(String text)`
    `void` | `writeTree(TreeNode node)`
    `void` | `writeTypeId(Object id)`
    `void` | `writeUTF8String(byte[] text, int offset, int length)`

    - ### Methods inherited from class com.fasterxml.jackson.core.JsonGenerator

      `_reportError, _throwInternal, _verifyOffsets, _writeSimpleObject, canOmitFields, canUseSchema, canWriteFormattedNumbers, configure, getCharacterEscapes, getCurrentValue, getFormatFeatures, getHighestEscapedChar, getOutputBuffered, getOutputTarget, getPrettyPrinter, getSchema, overrideFormatFeatures, setCharacterEscapes, setCurrentValue, setHighestNonEscapedChar, setPrettyPrinter, setRootValueSeparator, setSchema, writeArray, writeArray, writeArray, writeArrayFieldStart, writeBinary, writeBinary, writeBinary, writeBinaryField, writeBooleanField, writeFieldId, writeNullField, writeNumberField, writeNumberField, writeNumberField, writeNumberField, writeNumberField, writeObjectField, writeObjectFieldStart, writeObjectRef, writeOmittedField, writeRawValue, writeStartArray, writeStringField`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_GENERATOR_FEATURES

          protected static final int DEFAULT_GENERATOR_FEATURES

    - #### _objectCodec

          protected ObjectCodec _objectCodec

      Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.

    - #### _generatorFeatures

          protected int _generatorFeatures

      Bit flag composed of bits that indicate which `JsonGenerator.Feature`s are enabled.

      NOTE: most features have no effect on this class

    - #### _closed

          protected boolean _closed

    - #### _hasNativeTypeIds

          protected boolean _hasNativeTypeIds

      Since:
        2.3

    - #### _hasNativeObjectIds

          protected boolean _hasNativeObjectIds

      Since:
        2.3

    - #### _mayHaveNativeIds

          protected boolean _mayHaveNativeIds

      Since:
        2.3

    - #### _forceBigDecimal

          protected boolean _forceBigDecimal

      Flag set during construction, if use of `BigDecimal` is to be forced on all floating-point values.
      Since:
        2.7

    - #### _first

          protected TokenBuffer.Segment _first

      First segment, for contents this buffer has

    - #### _last

          protected TokenBuffer.Segment _last

      Last segment of this buffer, one that is used for appending more tokens

    - #### _appendAt

          protected int _appendAt

      Offset within last segment,

    - #### _typeId

          protected Object _typeId

      If native type ids supported, this is the id for following value (or first token of one) to be written.

    - #### _objectId

          protected Object _objectId

      If native object ids supported, this is the id for following value (or first token of one) to be written.

    - #### _hasNativeId

          protected boolean _hasNativeId

      Do we currently have a native type or object id buffered?

    - #### _writeContext

          protected JsonWriteContext _writeContext

  - ### Constructor Detail

    - #### TokenBuffer

          @Deprecated
          public TokenBuffer(ObjectCodec codec)

      Deprecated. since 2.3 preferred variant is one that takes `JsonParser` or additional boolean parameter.
      Parameters:
        `codec` - Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.

    - #### TokenBuffer

          public TokenBuffer(ObjectCodec codec,
                     boolean hasNativeIds)

      Parameters:
        `codec` - Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.
        `hasNativeIds` - Whether resulting `JsonParser` (if created) is considered to support native type and object ids

    - #### TokenBuffer

          public TokenBuffer(JsonParser p)

      Since:
        2.3

    - #### TokenBuffer

          public TokenBuffer(JsonParser p,
                     DeserializationContext ctxt)

      Since:
        2.7

  - ### Method Detail

    - #### forceUseOfBigDecimal

          public TokenBuffer forceUseOfBigDecimal(boolean b)

      Since:
        2.7

    - #### version

          public Version version()

      Specified by:
        `version` in interface `Versioned`
      Specified by:
        `version` in class `JsonGenerator`

    - #### asParser

          public JsonParser asParser()

      Method used to create a `JsonParser` that can read contents stored in this buffer. Will use default `_objectCodec` for object conversions.

      Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

      Returns:
        Parser that can be used for reading contents stored in this buffer

    - #### asParser

          public JsonParser asParser(ObjectCodec codec)

      Method used to create a `JsonParser` that can read contents stored in this buffer.

      Note: instances are not synchronized, that is, they are not thread-safe if there are concurrent appends to the underlying buffer.

      Parameters:
        `codec` - Object codec to use for stream-based object conversion through parser/generator interfaces. If null, such methods can not be used.
      Returns:
        Parser that can be used for reading contents stored in this buffer

    - #### asParser

          public JsonParser asParser(JsonParser src)

      Parameters:
        `src` - Parser to use for accessing source information like location, configured codec

    - #### firstToken

          public JsonToken firstToken()

    - #### append

          public TokenBuffer append(TokenBuffer other)
                             throws IOException

      Helper method that will append contents of given buffer into this buffer. Not particularly optimized; can be made faster if there is need.
      Returns:
        This buffer
      Throws:
        `IOException`

    - #### serialize

          public void serialize(JsonGenerator gen)
                         throws IOException

      Helper method that will write all contents of this buffer using given `JsonGenerator`.

      Note: this method would be enough to implement `JsonSerializer` for `TokenBuffer` type; but we can not have upwards references (from core to mapper package); and as such we also can not take second argument.

      Throws:
        `IOException`

    - #### deserialize

          public TokenBuffer deserialize(JsonParser p,
                                DeserializationContext ctxt)
                                  throws IOException

      Helper method used by standard deserializer.
      Throws:
        `IOException`
      Since:
        2.3

    - #### toString

          public String toString()

      Overrides:
        `toString` in class `Object`

    - #### enable

          public JsonGenerator enable(JsonGenerator.Feature f)

      Specified by:
        `enable` in class `JsonGenerator`

    - #### disable

          public JsonGenerator disable(JsonGenerator.Feature f)

      Specified by:
        `disable` in class `JsonGenerator`

    - #### isEnabled

          public boolean isEnabled(JsonGenerator.Feature f)

      Specified by:
        `isEnabled` in class `JsonGenerator`

    - #### getFeatureMask

          public int getFeatureMask()

      Specified by:
        `getFeatureMask` in class `JsonGenerator`

    - #### setFeatureMask

          @Deprecated
          public JsonGenerator setFeatureMask(int mask)

      Deprecated.
      Specified by:
        `setFeatureMask` in class `JsonGenerator`

    - #### overrideStdFeatures

          public JsonGenerator overrideStdFeatures(int values,
                                          int mask)

      Overrides:
        `overrideStdFeatures` in class `JsonGenerator`

    - #### useDefaultPrettyPrinter

          public JsonGenerator useDefaultPrettyPrinter()

      Specified by:
        `useDefaultPrettyPrinter` in class `JsonGenerator`

    - #### setCodec

          public JsonGenerator setCodec(ObjectCodec oc)

      Specified by:
        `setCodec` in class `JsonGenerator`

    - #### getCodec

          public ObjectCodec getCodec()

      Specified by:
        `getCodec` in class `JsonGenerator`

    - #### getOutputContext

          public final JsonWriteContext getOutputContext()

      Specified by:
        `getOutputContext` in class `JsonGenerator`

    - #### canWriteBinaryNatively

          public boolean canWriteBinaryNatively()

      Since we can efficiently store `byte[]`, yes.
      Overrides:
        `canWriteBinaryNatively` in class `JsonGenerator`

    - #### flush

          public void flush()
                     throws IOException

      Specified by:
        `flush` in interface `Flushable`
      Specified by:
        `flush` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### close

          public void close()
                     throws IOException

      Specified by:
        `close` in interface `Closeable`
      Specified by:
        `close` in interface `AutoCloseable`
      Specified by:
        `close` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### isClosed

          public boolean isClosed()

      Specified by:
        `isClosed` in class `JsonGenerator`

    - #### writeStartArray

          public final void writeStartArray()
                                     throws IOException

      Specified by:
        `writeStartArray` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeEndArray

          public final void writeEndArray()
                                   throws IOException

      Specified by:
        `writeEndArray` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeStartObject

          public final void writeStartObject()
                                      throws IOException

      Specified by:
        `writeStartObject` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeStartObject

          public void writeStartObject(Object forValue)
                                throws IOException

      Overrides:
        `writeStartObject` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeEndObject

          public final void writeEndObject()
                                    throws IOException

      Specified by:
        `writeEndObject` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeFieldName

          public final void writeFieldName(String name)
                                    throws IOException

      Specified by:
        `writeFieldName` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeFieldName

          public void writeFieldName(SerializableString name)
                              throws IOException

      Specified by:
        `writeFieldName` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeString

          public void writeString(String text)
                           throws IOException

      Specified by:
        `writeString` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeString

          public void writeString(char[] text,
                         int offset,
                         int len)
                           throws IOException

      Specified by:
        `writeString` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeString

          public void writeString(SerializableString text)
                           throws IOException

      Specified by:
        `writeString` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRawUTF8String

          public void writeRawUTF8String(byte[] text,
                                int offset,
                                int length)
                                  throws IOException

      Specified by:
        `writeRawUTF8String` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeUTF8String

          public void writeUTF8String(byte[] text,
                             int offset,
                             int length)
                               throws IOException

      Specified by:
        `writeUTF8String` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRaw

          public void writeRaw(String text)
                        throws IOException

      Specified by:
        `writeRaw` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRaw

          public void writeRaw(String text,
                      int offset,
                      int len)
                        throws IOException

      Specified by:
        `writeRaw` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRaw

          public void writeRaw(SerializableString text)
                        throws IOException

      Overrides:
        `writeRaw` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRaw

          public void writeRaw(char[] text,
                      int offset,
                      int len)
                        throws IOException

      Specified by:
        `writeRaw` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRaw

          public void writeRaw(char c)
                        throws IOException

      Specified by:
        `writeRaw` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRawValue

          public void writeRawValue(String text)
                             throws IOException

      Specified by:
        `writeRawValue` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRawValue

          public void writeRawValue(String text,
                           int offset,
                           int len)
                             throws IOException

      Specified by:
        `writeRawValue` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeRawValue

          public void writeRawValue(char[] text,
                           int offset,
                           int len)
                             throws IOException

      Specified by:
        `writeRawValue` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(short i)
                           throws IOException

      Overrides:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(int i)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(long l)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(double d)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(float f)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(BigDecimal dec)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(BigInteger v)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNumber

          public void writeNumber(String encodedValue)
                           throws IOException

      Specified by:
        `writeNumber` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeBoolean

          public void writeBoolean(boolean state)
                            throws IOException

      Specified by:
        `writeBoolean` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeNull

          public void writeNull()
                         throws IOException

      Specified by:
        `writeNull` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeObject

          public void writeObject(Object value)
                           throws IOException

      Specified by:
        `writeObject` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeTree

          public void writeTree(TreeNode node)
                         throws IOException

      Specified by:
        `writeTree` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeBinary

          public void writeBinary(Base64Variant b64variant,
                         byte[] data,
                         int offset,
                         int len)
                           throws IOException

      Specified by:
        `writeBinary` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### writeBinary

          public int writeBinary(Base64Variant b64variant,
                        InputStream data,
                        int dataLength)

      Although we could support this method, it does not necessarily make sense: we can not make good use of streaming because buffer must hold all the data. Because of this, currently this will simply throw `UnsupportedOperationException`
      Specified by:
        `writeBinary` in class `JsonGenerator`

    - #### canWriteTypeId

          public boolean canWriteTypeId()

      Overrides:
        `canWriteTypeId` in class `JsonGenerator`

    - #### canWriteObjectId

          public boolean canWriteObjectId()

      Overrides:
        `canWriteObjectId` in class `JsonGenerator`

    - #### writeTypeId

          public void writeTypeId(Object id)

      Overrides:
        `writeTypeId` in class `JsonGenerator`

    - #### writeObjectId

          public void writeObjectId(Object id)

      Overrides:
        `writeObjectId` in class `JsonGenerator`

    - #### writeEmbeddedObject

          public void writeEmbeddedObject(Object object)
                                   throws IOException

      Overrides:
        `writeEmbeddedObject` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### copyCurrentEvent

          public void copyCurrentEvent(JsonParser p)
                                throws IOException

      Overrides:
        `copyCurrentEvent` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### copyCurrentStructure

          public void copyCurrentStructure(JsonParser p)
                                    throws IOException

      Overrides:
        `copyCurrentStructure` in class `JsonGenerator`
      Throws:
        `IOException`

    - #### _append

          protected final void _append(JsonToken type)

    - #### _append

          protected final void _append(JsonToken type,
                     Object value)

    - #### _appendValue

          protected final void _appendValue(JsonToken type)

      Similar to `_append(JsonToken)` but also updates context with knowledge that a scalar value was written
      Since:
        2.6.4

    - #### _appendValue

          protected final void _appendValue(JsonToken type,
                          Object value)

      Similar to `_append(JsonToken,Object)` but also updates context with knowledge that a scalar value was written
      Since:
        2.6.4

    - #### _appendRaw

          protected final void _appendRaw(int rawType,
                        Object value)

    - #### _reportUnsupportedOperation

          protected void _reportUnsupportedOperation()

      Overrides:
        `_reportUnsupportedOperation` in class `JsonGenerator`

Copyright © 2008–2016 FasterXML. All rights reserved.