Protocol buffers are Google's language-neutral, platform-neutral, extensible mechanism for serializing structured data -- think XML, but smaller, faster, and simpler. You define how you want your data to be structured once. This takes the form of a template that describes the data structure. You use this template to encode and decode your data structure into wire-streams that may be sent-to or read-from your peers. The underlying wire stream is platform independent, lossless, and may be used to interwork with a variety of languages and systems regardless of word size or endianness. Techniques exist to safely extend your data structure without breaking deployed programs that are compiled against the "old" format.
The idea behind Google's Protocol Buffers is that you define your
structured messages using a domain-specific language and tool set.
Further documentation on this is at
https://developers.google.com/protocol-buffers.
There are two ways you can use protobufs in Prolog:
.proto file: protobuf_parse_from_codes/3
and
protobuf_serialize_to_codes/3.
The protobuf_parse_from_codes/3
and protobuf_serialize_to_codes/3
interface translates between a "wire stream" and a Prolog term. This
interface takes advantage of SWI-Prolog's
dict. There is a protoc
plugin (protoc-gen-swipl) that generates a Prolog file of
meta-information that captures the .proto file's definition
in the protobufs module:
proto_meta_normalize(Unnormalized, Normalized)proto_meta_package(Package, FileName, Options)proto_meta_message_type( Fqn, Package, Name)proto_meta_message_type_map_entry( Fqn)proto_meta_field_name( Fqn, FieldNumber, FieldName, FqnName)proto_meta_field_json_name( FqnName, JsonName)proto_meta_field_label( FqnName, LabelRepeatOptional) % 'LABEL_OPTIONAL', 'LABEL_REQUIRED', 'LABEL_REPEATED'proto_meta_field_type( FqnName, Type) % 'TYPE_INT32', 'TYPE_MESSAGE', etcproto_meta_field_type_name( FqnName, TypeName)proto_meta_field_default_value( FqnName, DefaultValue)proto_meta_field_option_packed( FqnName)proto_meta_enum_type( FqnName, Fqn, Name)proto_meta_enum_value( FqnName, Name, Number)proto_meta_field_oneof_index( FqnName, Index)proto_meta_oneof( FqnName, Index, Name)
The protobuf_message/2
interface allows you to define your message template as a list of
predefined Prolog terms that correspond to production rules in the
Definite Clause Grammar (DCG) that realizes the interpreter. Each
production rule has an equivalent rule in the protobuf grammar. The
process is not unlike specifiying the format of a regular expression. To
encode a template to a wire-stream, you pass a grounded template, X,
and variable, Y, to
protobuf_message/2. To
decode a wire-stream, Y, you pass an ungrounded template, X,
along with a grounded wire-stream, Y, to
protobuf_message/2. The
interpreter will unify the unbound variables in the template with values
decoded from the wire-stream.
For an overview and tutorial with examples, see
library(protobufs): Google's Protocol Buffers (section
1) Examples of usage may also be found by inspecting
test_protobufs.pl
and the
demo
directory, or by looking at the "addressbook" example that is typically
installed at /usr/lib/swi-prolog/doc/packages/examples/protobufs/interop/addressbook.pl
MessageType), creating a Prolog term.
Protoc must have been run (with the --swipl_out=
option and the resulting top-level _pb.pl file loaded. For
more details, see the "protoc" section of the overview documentation.
Fails if the message can't be parsed or if the appropriate meta-data
from protoc hasn't been loaded.
All fields that are omitted from the WireCodes are set
to their default values (typically the empty string or 0, depending on
the type; or [] for repeated groups). There is no way of
testing whether a value was specified in WireCodes or given
its default value (that is, there is no equivalent of the Python
implementation's =HasField`). Optional embedded messages and groups do
not have any default value -- you must check their existence by using get_dict/3
or similar. If a field is part of a "oneof" set, then none of the other
fields is set. You can determine which field had a value by using get_dict/3.
| WireCodes | Wire format of the message from
e.g., read_stream_to_codes/2. (The stream
should have options encoding(octet) and type(binary),
either as options to read_file_to_codes/3
or by calling set_stream/2 on the stream
to read_stream_to_codes/2.) |
| MessageType | Fully qualified message name
(from the .proto file's package and message).
For example, if the package is google.protobuf
and the message is FileDescriptorSet, then you would use
'.google.protobuf.FileDescriptorSet' or 'google.protobuf.FileDescriptorSet'.
If there's no package name, use e.g.: 'MyMessage or '.MyMessage'.
You can see the packages by looking at
protobufs:proto_meta_package(Pkg,File,_) and the message
names and fields by
protobufs:proto_meta_field_name('.google.protobuf.FileDescriptorSet', FieldNumber, FieldName, FqnName)
(the initial’.' is not optional for these facts, only for the
top-level name given to protobuf_serialize_to_codes/3). |
| Term | The generated term, as nested dicts. |
version_error(Module-Version) you need to recompile the Module
with a newer version of protoc.library(protobufs): Google's Protocol Buffers (section
1.3.1).proto extensions. map fields don't get special treatment (but see protobuf_map_pairs/3). library(http/json) and json_read_dict/3) true and value_string_as
(similar to json_read_dict/3) MessageType).
Protoc must have been run (with the --swipl_out=
option and the resulting top-level _pb.pl file loaded. For
more details, see the "protoc" section of the overview documentation.
Fails if the term isn't of an appropriate form or if the appropriate
meta-data from protoc hasn't been loaded, or if a field
name is incorrect (and therefore nothing in the meta-data matches it).
| Term | The Prolog form of the data, as nested dicts. |
| MessageType | Fully qualified message name
(from the .proto file's package and message).
For example, if the package is google.protobuf
and the message is FileDescriptorSet, then you would use
'.google.protobuf.FileDescriptorSet' or 'google.protobuf.FileDescriptorSet'.
If there's no package name, use e.g.: 'MyMessage or '.MyMessage'.
You can see the packages by looking at
protobufs:proto_meta_package(Pkg,File,_) and the message
names and fields by
protobufs:proto_meta_field_name('.google.protobuf.FileDescriptorSet', FieldNumber, FieldName, FqnName)
(the initial’.' is not optional for these facts, only for the
top-level name given to protobuf_serialize_to_codes/3). |
| WireCodes | Wire format of the message, which
can be output using
format('~s', [WireCodes]). |
version_error(Module-Version) you need to recompile the Module
with a newer version of protoc. library(protobufs): Google's Protocol Buffers (section
1.3.1)map fields don't get special treatment (but see protobuf_map_pairs/3). oneof is not checked for validity.| Template | is a protobuf grammar specification. On decode, unbound variables in the Template are unified with their respective values in the WireStream. On encode, Template must be ground. |
| WireStream | is a code list that was generated by a protobuf encoder using an equivalent template. |
MessageType’s FieldName is
defined as a map<...> in the .proto file.DictTag{key:Key, value:Value} and a key-value list as
described in library(pairs). At least one of ProtobufTermList
and Pairs must be instantiated; DictTag can be
uninstantiated. If
ProtobufTermList is from a term created by
protobuf_parse_from_codes/3,
the ordering of the items is undefined; you can order them by using keysort/2
(or by a predicate such as
dict_pairs/3, list_to_assoc/2,
or list_to_rbtree/2.