XrdHttpReadRangeHandler Class Reference

#include <XrdHttpReadRangeHandler.hh>

Collaboration diagram for XrdHttpReadRangeHandler:

Classes
struct	Configuration
struct	Error
struct	UserRange

Public Types
typedef std::vector< UserRange >	UserRangeList

Public Member Functions
	XrdHttpReadRangeHandler (const Configuration &conf)
const Error &	getError () const
	return the Error object
size_t	getMaxRanges () const
	return the maximum number of ranges that may be requested
bool	isFullFile ()
	indicates when there were no valid Range head ranges supplied
bool	isSingleRange ()
	indicates a single range (implied whole file, or single range) or empty file
const UserRangeList &	ListResolvedRanges ()
	return resolved (i.e. obsolute start and end) byte ranges desired
const XrdHttpIOList &	NextReadList ()
	return XrdHttpIOList for sending to read or readv
void	NotifyError ()
	Force handler to enter error state.
int	NotifyReadResult (const ssize_t ret, const UserRange **const urp, bool &start, bool &allend)
	Advance internal counters concerning received bytes.
void	ParseContentRange (const char *const line)
	parse the line after a "Range: " http request header
void	reset ()
	resets this handler
int	SetFilesize (const off_t sz)
	sets the filesize, used during resolving and issuing range requests

Static Public Member Functions
static int	Configure (XrdSysError &Eroute, const char *const parms, Configuration &cfg)

Static Public Attributes
static constexpr size_t	READV_MAXCHUNKS = 512
static constexpr size_t	READV_MAXCHUNKSIZE = 512*1024
static constexpr size_t	RREQ_MAXSIZE = 8*1024*1024

Detailed Description

Class responsible for parsing the HTTP Content-Range header coming from the client, generating appropriate read ranges for read or readv and tracking the responses to the requests.

Definition at line 36 of file XrdHttpReadRangeHandler.hh.

Member Typedef Documentation

UserRangeList

typedef std::vector<UserRange> XrdHttpReadRangeHandler::UserRangeList

Definition at line 103 of file XrdHttpReadRangeHandler.hh.

Constructor & Destructor Documentation

XrdHttpReadRangeHandler()

XrdHttpReadRangeHandler::XrdHttpReadRangeHandler ( const Configuration & conf )

inline

Constructor. Supplied with an Configuration object. The supplied object remains owned by the caller, but should remain valid throughout the lifetime of the ReadRangeHandler.

Parameters

conf Configuration object.

Definition at line 113 of file XrdHttpReadRangeHandler.hh.

  {
    rRequestMaxBytes_       = RREQ_MAXSIZE;
    vectorReadMaxChunkSize_ = READV_MAXCHUNKSIZE;
    vectorReadMaxChunks_    = READV_MAXCHUNKS;
 
    if( conf.haveSizes )
    {
      vectorReadMaxChunkSize_ = conf.readv_ior_max;
      vectorReadMaxChunks_    = conf.readv_iov_max;
      rRequestMaxBytes_       = conf.reqs_max;
    }
    reset();
  }

References XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, READV_MAXCHUNKS, READV_MAXCHUNKSIZE, XrdHttpReadRangeHandler::Configuration::reqs_max, reset(), and RREQ_MAXSIZE.

Here is the call graph for this function:

Member Function Documentation

Configure()

int XrdHttpReadRangeHandler::Configure ( XrdSysError & Eroute, const char *const parms, Configuration & cfg )

static

Parses a configuration into a Configuration object.

Parameters

Eroute Error reporting object

Parameters

parms Configuration string.

Parameters

cfg an output Configuration object

Returns

0 for success, otherwise failure.

static, class method: initialise a configuraiton object. parms is currently only content of environment variable XRD_READV_LIMITS, to get the specific kXR_readv limits.

Definition at line 42 of file XrdHttpReadRangeHandler.cc.

{
  if( !parms ) return 0;
 
  std::vector<std::string> splitArgs;
  XrdOucTUtils::splitString( splitArgs, parms, "," );
  if( splitArgs.size() < 2 ) return 0;
 
  //----------------------------------------------------------------------------
  // params is expected to be "<readv_ior_max>,<readv_iov_max>"
  //----------------------------------------------------------------------------
  std::string iorstr = splitArgs[0];
  std::string iovstr = splitArgs[1];
  XrdOucUtils::trim( iorstr );
  XrdOucUtils::trim( iovstr );
 
  int val;
  if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_ior_max",
                      iorstr.c_str(), &val, 1, -1 ) )
  {
    return -1;
  }
 
  cfg.readv_ior_max = val;
  if( XrdOuca2x::a2i( Eroute, "Error reading specific value of readv_iov_max",
                      iovstr.c_str(), &val, 1, -1 ) )
  {
    return -1;
  }
 
  cfg.readv_iov_max = val;
  cfg.reqs_max      = RREQ_MAXSIZE;
  cfg.haveSizes     = true;
 
  return 0;
}

References XrdOuca2x::a2i(), XrdHttpReadRangeHandler::Configuration::haveSizes, XrdHttpReadRangeHandler::Configuration::readv_ior_max, XrdHttpReadRangeHandler::Configuration::readv_iov_max, XrdHttpReadRangeHandler::Configuration::reqs_max, RREQ_MAXSIZE, XrdOucTUtils::splitString(), and XrdOucUtils::trim().

Here is the call graph for this function:

getError()

const XrdHttpReadRangeHandler::Error & XrdHttpReadRangeHandler::getError

(

)

const

return the Error object

getter for the Error object. The object can be inspected with its operator bool() method to indicate an error has happened. Error code and message are available in other members of Error.

Definition at line 86 of file XrdHttpReadRangeHandler.cc.

{
  return error_;
}

getMaxRanges()

size_t XrdHttpReadRangeHandler::getMaxRanges

(

)

const

return the maximum number of ranges that may be requested

Return the maximum number of expected ranges for the request. Used to determine whether a vector range request may be used at open time. At that time, isSingleRange() cannot be invoked because it internally resolves the correct ranges which requires the file size to be set (which only occurs after file-open). The use case is to determine whether we should set the sequential I/O flag at open time.

Definition at line 102 of file XrdHttpReadRangeHandler.cc.

{
  return rawUserRanges_.empty() ? 1 : rawUserRanges_.size();
}

isFullFile()

bool XrdHttpReadRangeHandler::isFullFile

(

)

indicates when there were no valid Range head ranges supplied

Indicates no valid Range header was given and thus the implication is that whole file is required. A range or ranges may be given that cover the whole file but that situation is not detected.

Definition at line 94 of file XrdHttpReadRangeHandler.cc.

{
  return rawUserRanges_.empty();
}

isSingleRange()

bool XrdHttpReadRangeHandler::isSingleRange

(

)

indicates a single range (implied whole file, or single range) or empty file

Incidcates whether there is a single range, either given by a Range header with single range or implied by having no Range header. Also returns true for an empty file, although there is no range of bytes.

Returns

true if there is a single range.

Definition at line 110 of file XrdHttpReadRangeHandler.cc.

{
  if( !rangesResolved_ )
    resolveRanges();
 
  return( resolvedUserRanges_.size() <= 1 );
}

ListResolvedRanges()

const XrdHttpReadRangeHandler::UserRangeList & XrdHttpReadRangeHandler::ListResolvedRanges

(

)

return resolved (i.e. obsolute start and end) byte ranges desired

Returns a reference of the list of ranges. These are resolved, meaning that if there was no Range header, or it was in the form -N or N-, the file size is used to compute the actual range of bytes that are needed. The list remains owned by the handler and may be invalidated on reset().

Returns

List of ranges in a UserRangeList object. The returned list may be empty, i.e. for an empty file or if there is an error. Use getError() to see if there is an error.

Definition at line 121 of file XrdHttpReadRangeHandler.cc.

{
  static const UserRangeList emptyList;
 
  if( !rangesResolved_ )
    resolveRanges();
 
  if( error_ )
    return emptyList;
 
  return resolvedUserRanges_;
}

NextReadList()

const XrdHttpIOList & XrdHttpReadRangeHandler::NextReadList

(

)

return XrdHttpIOList for sending to read or readv

Requests a XrdHttpIOList (vector of XrdOucIOVec2) that describes the next bytes that need to be fetched from a file. If there is more than one chunk it is size appropriately for a readv request, if there is one request it should be sent as a read request. Therefore the chunks do not necessarily correspond to the ranges the user requested. The caller issue the requests in the order provided and call NotifyReadResult with the ordered results.

Returns

a reference to a XrdHttpIOList. The object remains owned by the handler. It may be invalided by a new call to NextReadList() or reset(). The returned list may be empty, which implies no more reads are needed One can use getError() to see if there is an error.

Definition at line 137 of file XrdHttpReadRangeHandler.cc.

{
  static const XrdHttpIOList emptyList;
 
  if( !rangesResolved_ )
    resolveRanges();
 
  if( error_ )
    return emptyList;
 
  if( !splitRange_.empty() )
  {
    if( currSplitRangeIdx_ == 0 && currSplitRangeOff_ == 0 )
    {
      //------------------------------------------------------------------------
      // Nothing read: Prevent scenario where data is expected but none is
      // actually read E.g. Accessing files which return the results of a script
      //------------------------------------------------------------------------
      error_.set( 500, "Stopping request because more data is expected "
                       "but no data has been read." );
      return emptyList;
    }
 
    //--------------------------------------------------------------------------
    // we may have some unacknowledged portion of the last range; maybe due to a
    // short read. so remove what was received and potentially reissue.
    //--------------------------------------------------------------------------
 
    trimSplit();
    if( !splitRange_.empty() )
      return splitRange_;
  }
 
  if( splitRangeIdx_ >= resolvedUserRanges_.size() )
    return emptyList;
 
  splitRanges();
 
  return splitRange_;
}

NotifyError()

void XrdHttpReadRangeHandler::NotifyError

(

)

Force handler to enter error state.

Force the handler to enter error state. Sets a generic error message if there was not already an error.

Definition at line 181 of file XrdHttpReadRangeHandler.cc.

{
  if( error_ )
    return;
 
  error_.set( 500, "An error occurred." );
}

NotifyReadResult()

int XrdHttpReadRangeHandler::NotifyReadResult	(	const ssize_t	ret,
		const UserRange **const	urp,
		bool &	start,
		bool &	allend )

Advance internal counters concerning received bytes.

Notifies the handler about the arrival of bytes from a read or readv request. The handler tracks the progress of the arriving bytes against the bytes ranges the user requested.

Parameters

ret	the number of bytes received
urp	a pointer to a pointer of a UserRange object. If urp is not nullptr, the pointer to a UserRange is returned that describes the current range associated with the received bytes. The handler retains ownership of the returned object. reset() of the handler invalidates the UserRange object.
start	is an output bool parameter that indicates whether the received bytes mark the start of a UserRange.
allend	is an output bool parameter that indicates whether the received bytes mark the end of all the UserRanges

Returns

0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 192 of file XrdHttpReadRangeHandler.cc.

{
  if( error_ )
    return -1;
 
  if( ret == 0 )
    return 0;
 
  if( ret < 0 )
  {
    error_.set( 500, "Range handler read failure." );
    return -1;
  }
 
  if( !rangesResolved_ )
  {
    error_.set( 500, "Range handler ranges not yet resolved." );
    return -1;
  }
 
  if( splitRange_.empty() )
  {
    error_.set( 500, "No ranges being read." );
    return -1;
  }
 
  start  = false;
  allend = false;
 
  if( currSplitRangeIdx_ >= splitRange_.size() ||
      resolvedRangeIdx_  >= resolvedUserRanges_.size()  )
  {
    error_.set( 500, "Range handler index invalid." );
    return -1;
  }
 
  if( urp )
    *urp = &resolvedUserRanges_[resolvedRangeIdx_];
 
  if( resolvedRangeOff_ == 0 )
    start = true;
 
  const int   clen  = splitRange_[currSplitRangeIdx_].size;
 
  const off_t ulen  = resolvedUserRanges_[resolvedRangeIdx_].end
                        - resolvedUserRanges_[resolvedRangeIdx_].start + 1;
 
  currSplitRangeOff_ += ret;
  resolvedRangeOff_  += ret;
 
  if( currSplitRangeOff_  > clen || resolvedRangeOff_ > ulen )
  {
    error_.set( 500, "Range handler read crossing chunk boundary." );
    return -1;
  }
 
  if( currSplitRangeOff_ == clen )
  {
    currSplitRangeOff_ = 0;
    currSplitRangeIdx_++;
 
    if( currSplitRangeIdx_ >= splitRange_.size() )
    {
      currSplitRangeIdx_ = 0;
      splitRange_.clear();
    }
  }
 
  if( resolvedRangeOff_ == ulen )
  {
    resolvedRangeIdx_++;
    resolvedRangeOff_ = 0;
    if( resolvedRangeIdx_ >= resolvedUserRanges_.size() )
      allend = true;
  }
 
  return 0;
}

References XrdHttpReadRangeHandler::UserRange::end.

ParseContentRange()

void XrdHttpReadRangeHandler::ParseContentRange

(

const char *const

line

)

parse the line after a "Range: " http request header

Parses the Content-Range header value and sets the ranges within the object.

Parameters

line	the line under the format "bytes=0-19, 25-30" In case the parsing fails any partial results are cleared. There is no error notification as the rest of the request processing should continue in any case.

Definition at line 280 of file XrdHttpReadRangeHandler.cc.

{
  char *str1, *saveptr1, *token;
 
  std::unique_ptr< char, decltype(std::free)* >
    line_copy { strdup( line ), std::free };
 
  //----------------------------------------------------------------------------
  // line_copy is argument of the Range header.
  //
  // e.g. "bytes=15-17,20-25"
  // We skip the unit prefix (upto first '='). We don't
  // enforce this prefix nor check what it is (e.g. 'bytes')
  //----------------------------------------------------------------------------
 
  str1  = line_copy.get();
  token = strchr(str1,'=');
  if (token) str1 = token + 1;
 
  //----------------------------------------------------------------------------
  // break up the ranges and process each
  //----------------------------------------------------------------------------
 
  for( ; ; str1 = NULL )
  {
    token = strtok_r( str1, " ,\n\r", &saveptr1 );
    if( token == NULL )
      break;
 
    if( !strlen(token) ) continue;
 
    const int rc = parseOneRange( token );
    if( rc )
    {
      //------------------------------------------------------------------------
      // on error we ignore the whole range header
      //------------------------------------------------------------------------
      rawUserRanges_.clear();
      return;
    }
  }
}

reset()

void XrdHttpReadRangeHandler::reset

(

)

resets this handler

Resets the object state, ready for handling a new request.

Definition at line 326 of file XrdHttpReadRangeHandler.cc.

{
  error_.reset();
  rawUserRanges_.clear();
  rawUserRanges_.shrink_to_fit();
  resolvedUserRanges_.clear();
  resolvedUserRanges_.shrink_to_fit();
  splitRange_.clear();
  splitRange_.shrink_to_fit();
  rangesResolved_    = false;
  splitRangeIdx_     = 0;
  splitRangeOff_     = 0;
  currSplitRangeIdx_ = 0;
  currSplitRangeOff_ = 0;
  resolvedRangeIdx_  = 0;
  resolvedRangeOff_  = 0;
  filesize_          = 0;
}

Referenced by XrdHttpReadRangeHandler().

Here is the caller graph for this function:

SetFilesize()

int XrdHttpReadRangeHandler::SetFilesize

(

const off_t

sz

)

sets the filesize, used during resolving and issuing range requests

Notifies of the current file size. This information is required for processing range requests that imply reading to the end or a certain position before the end of a file. It is also used to determine when read or readv need no longer be issued when reading the whole file. Can be called once or more, after reset() but before isSingleRange(), ListResolvedRanges() or NextReadList() methods.

Parameters

sz	the size of the file

Returns

0 upon success, -1 if an error happened. One needs to call the getError() method to return the error.

Definition at line 348 of file XrdHttpReadRangeHandler.cc.

{
  if( error_ )
    return -1;
 
  if( rangesResolved_ )
  {
    error_.set( 500, "Filesize notified after ranges resolved." );
    return -1;
  }
 
  filesize_ = fs;
  return 0;
}

Member Data Documentation

READV_MAXCHUNKS

size_t XrdHttpReadRangeHandler::READV_MAXCHUNKS = 512

staticconstexpr

These are defaults for: READV_MAXCHUNKS Max length of the XrdHttpIOList vector. READV_MAXCHUNKSIZE Max length of a XrdOucIOVec2 element. RREQ_MAXSIZE Max bytes to issue in a whole readv/read.

Definition at line 45 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

READV_MAXCHUNKSIZE

size_t XrdHttpReadRangeHandler::READV_MAXCHUNKSIZE = 512*1024

staticconstexpr

Definition at line 46 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler().

RREQ_MAXSIZE

size_t XrdHttpReadRangeHandler::RREQ_MAXSIZE = 8*1024*1024

staticconstexpr

Definition at line 47 of file XrdHttpReadRangeHandler.hh.

Referenced by XrdHttpReadRangeHandler(), and Configure().

---

The documentation for this class was generated from the following files:

• XrdHttpReadRangeHandler.hh
• XrdHttpReadRangeHandler.cc