A Pliant stream is used to transfer some data as a set of bytes. The main usage is storing or loading data from disk, and sending or receiving data from a network connection.
Pliant moreover provides a lot of filters making compression, encryption, and applying standard protocols such as FTP, HTTP or SMTP straight forward.
Basic operations available on a stream are:
For experimented users, what is special about Pliant stream is just that the 'query' and 'configure' method deal with ASCII parameters as opposed to Posix 'ioctl'.
Files handling is covered in another article.
In order to append at the end of the file instead of overwriting the file content, you could use:
var ExtendedStatus status := s open "file:/tmp/test.bin" append+safe
Errors handling strategy
There are two possible policies:
When using the 'safe' policy, you can test at any time if some error occurred on the stream using '=failure' as in the very first provided sample. Moreover, when reading a stream where an error occurred, 'raw_read' will fill the buffer with zeros bytes, and 'readline' (see below) will return an empty line.
if s=success and s:close=success
Symmetrically, when reading a file, the single fact the data is non zero is sufficient to grant that everything went fine at open an previous readings steps.
A Pliant 'Stream' data type is not thread safe, so if you want two threads to work on the same stream, you have to protect the access using a semaphore.
There is one exception: if you open a stream with 'noautopost' flag set (see 'Advanced operations' section bellow for 'noautopost' semantic), then it is safe to have one thread reading from the stream, and another one writing to the stream.
Accessing a ASCII data stream
s writechars "Some text"
'writeline' will output the specified text, followed by a LF or CR+LF or CR end of line sequence.
s writeline "More text"
produces the same result as:
s writechars "More text"
while not s:atend
It is possible to rewind one line through:
s unreadline "Some text"
but it is deprecated feature: rather use the more powerfull 'rewind' feature bellow.
selecting end of line convention
When reading some content, the kind of end of line (LF, CR+LF or CR) will be auto discovered from the content, unless you specify it explicitly at open time.
s open "file:/tmp/test.txt" out+safe+cr+lf
Please notice that Pliant will default to using LF end of line even when running under Windows where the standard is CR+LF.
If the end of line convention might be unconsistent (change from a line to another) in the stream, you might use 'anyeol' flag to ask Pliant to check all possibilities with line:
s open "file:/tmp/test.txt" in+safe+anyeol
Accessing PML encoded data stream
PML is a kind of binary XML introduced and used by Pliant for most native network protocols.
File direct access
var Stream s
'open' has several variants with more or less parameters. The most complete one is:
The first parameter is the name, and the second is an options string.
We have already seen 'in' 'out' 'append' 'safe' and 'noautopost' open flags.
About filtering streams naming convention, if you write:
var ExtendedStatus status := s open "gzip:" "comment [dq]Some title[dq] level 1" out+safe pliant_default_file_system support
the 's' stream will transparently apply compression and feed compressed data to the 'support' provided underlying stream. Please notice that the underlying stream must be real Pliant object, created with:
var Link:Stream support :> new Stream
ovar Stream support
as opposed to:
var Stream support
The alternative way to create the filtering stream is:
var ExtendedStatus status := s open "gzip:file:/tmp/test.gz" "comment [dq]Some title[dq] level 1" out+safe pliant_default_file_system (null map Stream)
so that the underlying stream will transparently be created.
A more frequently uses variant of open is:
var ExtendedStatus status := s open "gzip:file:/tmp/test.gz" "comment [dq]Some title[dq] level 1" out+safe
and it is equivalent to:
var ExtendedStatus status := s open "[dq]gzip:file:/tmp/test.gz[dq] comment [dq]Some title[dq] level 1" out+safe
In other words, if the first character of the name parameter provided to open is a double quote, then the name will be slit to a real name plus an options string.
It is possible to store the content of the stream in memory, or pick it's content from memory:
Please remind that a blob contains just a set of bytes, and it's content can be set to map any area in memory:
var Address adr
This is documented in the 'Files handling' article:
var Str temp
You can force data in the stream our buffer to be sent to the disk or the network using 'flush' method:
s flush anytime
s flush async
s flush sync
'anytime' means that the operating system is free to keep the data in it's cache for some finit time before sending it.
If you do:
var Stream s
a 'flush async' instruction will be transparently inserted between the 'write' and the 'read' operation. I mean, when a read operation (readline or raw_read) does not find enough bytes already available in the input buffer, it checks the output buffer, and if some data are pending there, it forces them out.
s open "tcp://server/25" in+out+cr+lf+safe+noautopost
'read_available' is frequently used to speed up reading through having the application read data directly from the stream buffer has opposed to using 'raw_read' that copies data from the stream buffer to the application buffer:
s read_available (var Address adr) (var Int size)
s read_available (var Address adr) (var Int size) 100
When returning, 'adr' will be a pointer to the data available in the read buffer, and 'size' receives the number of available bytes. 'read_available' consumes the data.
A high function is provided to copy some data from a stream to another:
var Stream s1 s2
The third parameter is the minimal number of bytes to copy before returning, and the fourth the maximum. The actual number of bytes copied is returned, so either an error occured, or it will be in the specified range.
Pliant provides a mechanism to enable reading several time the same sequence in an input stream.
The rewindable data is stored, so consumes memory. So a 'rewind_limit' parameter exists to prevent consuming too much memory. The default value is 16MB.
s rewind_limit := 64*2^20
It would be a good idea to enable a two stages limit, so that when the first one is reached, data are dumped to a temporary file that can grow up to the second limit.
Testing the stream status
var CBool c := s is_open
Returns true if the last open operation succeeded and no close operation has been performed since.
var CBool c := s is_crashed
Returns true if some error happen in a read or a write operation on the stream.
s error "Some message"
Sets the crash bit of the stream. If the stream has not been open with 'safe' flag set, then a standard Pliant error with error ID error_id_io will be raised.
Clears the crash bit of the stream. Don't use it unless you reviewed the implied underlying Pliant code.
Please notice that:
var CBool c := s=success
is equivalent to:
var CBool c := s:is_open and not s:is_crashed
How streams machinery works
A Pliant stream uses an input and an output buffer.
Direct access to the read buffer
The read buffer is governed by the three fields 'read_buffer' 'read_cur' and 'read_stop'.
var Int avail := (cast s:stream_read_stop Int).-.(cast s:stream_read_cur Int)
Moeover, any application is allowed to change 'stream_read_cur' pointer, so if your applications know that 2 bytes are available, it can consume them through something like:
var uInt16 data := s:stream_read_cur map uInt16
Direct access to the write buffer
The write buffer is govered by the three fields 'write_buffer' 'write_cur' and 'write_stop'.
var Int room := (cast s:stream_write_stop Int).-.(cast s:stream_write_cur Int)
Moeover, any application is allowed to change 'stream_write_cur' pointer, so if your applications know that 2 bytes are available, it can set them through something like:
s:stream_write_cur map uInt16 := 5
Pliant file systems
A Pliant file system inherits from 'Filesystem' data type defined in /pliant/language/stream/filesystembase.pli module, with an extra 'open' method defined at the beginning of /pliant/language/stream/stream.pli module.
When 'open' stream method is called, the file system 'open' method will be called, and it is responsible to attach a stream driver to the stream. If open was provided few parameters 'pliant_default_file_system' is used, and it contains an object with 'MultiFileSystem' data type, defined in /pliant/language/stream/multi.pli
Pliant 'MultiFileSystem' is just a namespace dispatching filesystem. Basically, all file systems are recording themself to it using 'mount' method and providing a namespace header that just says: I'm handling files with name starting with 'xxx:'
A stream driver is a class implementing the API defined for 'StreamDriver' data type at the beginning of /pliant/language/stream/stream.pli module.
So, when defining a new kind of Stream, one declare a new class inheriting from 'StreamDriver', and a new class inheriting from 'FileSystem', then create a filesystem object (through a simple 'gvar' instruction) and record it in the default multiple file system namespace using 'mount' method.