Dell DX6000 Setup And Configuration Manual
  1. Manuals
  2. Brands
  3. Dell Manuals
  4. Software
  5. DX6000
  6. Setup and configuration manual

Dell DX6000 Setup And Configuration Manual

Dx content router setup and configuration guide version 2.2
Hide thumbs Also See for DX6000:
Table of Contents

Quick Links

See also: Hardware Owner's Manual, Administration Manual
Content Router Setup
and Configuration Guide
Version 2.2
Table of Contents
loading

Related Manuals for Dell DX6000

Summary of Contents for Dell DX6000

  • Page 1 Content Router Setup and Configuration Guide Version 2.2...
  • Page 2 Content Router Setup and Configuration Guide: Version 2.2 Copyright © 2010 Caringo, Inc. All Rights Reserved. No part of this document may be reproduced, transmitted, or transcribed without the written consent of Caringo, Inc.

  • Page 3: Table Of Contents

    Table of Contents 1. Introduction to DX Content Router ..................1 1.1. Overview of DX Content Router ................. 1 1.2. About this Document ....................1 1.2.1. Audience ....................... 1 1.2.2. Scope ......................1 2. Replication Topologies ......................2 2.1. Disaster Recovery ..................... 2 2.2.
  • Page 4 B.1. Publisher Configuration ................... 26 B.2. Replicator Configuration ..................30 C. Enumerator API ........................ 34 C.1. Enumerator Types ....................34 C.2. Enumerator Start ....................34 C.2.1. Enumerator Start Query Arguments .............. 35 C.2.2. Enumerator Start Response ................. 35 C.3. Enumerator Next ....................36 C.3.1.

  • Page 5: Introduction To Dx Content Router

    Chapter 1. Introduction to DX Content Router 1.1. Overview of DX Content Router A DX Storage cluster routinely replicates content objects to other nodes in the same cluster in order to improve fault tolerance and performance. For disaster recovery and other reasons, it may also be desirable to automatically replicate content objects to another, remote cluster.

  • Page 6: Replication Topologies

    Chapter 2. Replication Topologies A replication topology is a defined arrangement between independent DX Storage clusters, connected to one another via DX Content Router nodes. DX Content Router supports several alternative replication topologies. 2.1. Disaster Recovery DX Content Router allows an administrator to replicate some or all the streams stored in a primary cluster to a disaster recovery site.

  • Page 7: Content Distribution

    Below is a figure showing multiple primary clusters rolling up to one DR cluster. In order to distinguish the streams in Primary Cluster 1 from those in Primary Cluster 2, metadata needs to be stored with each stream identifying, at a minimum, the cluster of origin. 2.4.

  • Page 8: Dx Content Router Services

    Chapter 3. DX Content Router Services 3.1. Basic Architecture A DX Content Router lives within a DX Storage cluster but is always visible to other clusters, and perhaps to the external network at large. Consequently, it is assumed that all communication between DX Content Router services occurs over a secure TCP connection.

  • Page 9: Structure Of A Dx Content Router Node

    3.1.1. Structure of a DX Content Router Node DX Content Router consists of a server machine running Linux and executes one or both of two services: 1. Publisher - processes all streams stored in a cluster, filters them based on stream metadata, and publishes UUIDs to remote Subscribers.
  • Page 10 Similar to the previous proxy-enabled example, if the storage nodes are not able to communicate directly in the configured network topology, the Publisher can be configured to send responses via an SCSP Proxy in a mirrored configuration as well: Copyright © 2010 Caringo, Inc. Version 2.2 All rights reserved December 2010...

  • Page 11: Publisher Service

    The two optional DX Content Router services are designed and deployed as independent processes running on a server. We discuss each of these services in more detail in the following sections. 3.1.2. Publisher Service The Publisher service collects a comprehensive list of all the UUIDs stored in the cluster (as well as those that have been deleted), filters those UUIDs, and publishes the resulting lists of UUIDs to one or more remote Subscribers.
  • Page 12 The example above is a good starting point, but it, alone, will not perform the filtering required for this example. In order to select the PrimaryDR cluster as the destination of some of the locally stored streams, we want to find all streams whose content metadata contains a header called "DX Storage-priority"...
  • Page 13 clause. In the example above, where there are two publish clauses, all content streams can be queued for remote replication once for each publish. In addition, when there are two select clauses in a given publish clause, the content metadata is evaluated against each select clause’s filter set.

  • Page 14: Replicator Service

    • olderThan(dateSpec) - matches if the header value is a date and that date is older than the date given, which may be either an absolute date or a relative (to execution time) date. • intValue(int) – matches if the header value is an integer and executes the specified comparison against that integer (greater than, less than, equal to, etc).

  • Page 15: Installation

    Chapter 4. Installation DX Content Router installs as a service on a standard server, using standard Linux installation and package management utilities as outlined in the sections below. Note, that if running DX Content Router from a Cluster Services Node (CSN), the required infrastructure setup, installation and configuration are performed automatically as part of CSN configuration so the steps below are not necessary or even supported in some cases.
  • Page 16 bytes/sec between clusters based on the calculation above will need to modify the 'minBytesPerSec' parameter in the DX Storage node.cfg or common.cfg file to account for a slower rate and avoid timeouts. For example, to lower DX Storage's expectations for transfer rate to 512 bytes/sec, the following parameter should be added to the node or cluster level configuration file to override the default: minBytesPerSec = 512...

  • Page 17: Installing And Configuring Dx Content Router Services

    VLAN=yes 4. Start the new interface: ifup ifcfg-eth1:1 Note, that if the DX Content Router services are installed on a Cluster Services Node (CSN), creation of the IP addresses is performed automatically when the services are configured. 4.2. Installing and Configuring DX Content Router Services 4.2.1.

  • Page 18: Upgrading Dx Content Router

    4.3. Upgrading DX Content Router 4.3.1. Upgrading from 1.x Installations with a previously installed Ubuntu-based version of DX Content Router have two upgrade options. Administrators with replication downtime tolerance may choose to stop all DX Content Router processes, install Red Hat Enterprise Linux (RHEL) over Ubuntu on the existing DX Content Router Publisher and Replicator servers and then install and start both DX Content Router services again on both servers.

  • Page 19: Running And Managing Dx Content Router

    Chapter 5. Running and Managing DX Content Router 5.1. Starting DX Content Router Services DX Content Router Publisher and Replicator will attempt to start every time the server on which they are installed boots. Admins should be sure to update the config files for Publisher and/or Replicator after installation to ensure DX Content Router has the necessary information to start correctly.

  • Page 20: Publish All Streams On Two Separate Channels

    5.3.2. Publish all streams on two separate channels ...
  • Page 21 olderThan('Tue, 16 Oct 2007 00:00:00 GMT') 5.3.4.3. NotExists Clause 5.3.4.4. All Clauses 5.4. Using the Publisher Console The Publisher Console allows users to view progress made by the Publisher and the associated subscribers.

  • Page 23: Publisher Console Layout

    5.4.1. Publisher Console layout: • Version: the version of DX Content Router installed on the Publisher node • Uptime: the amount of time elapsed since the DX Content Router Publisher service was last started • Source Cluster: the group multicast address of the DX Storage cluster from which the Publisher is gathering UUIDs •...
  • Page 24 • Working: the subscriber is available and actively processing streams • Offline: the subscriber has not contacted the DX Content Router Publisher node within the expected configurable frequency • Idle: the subscriber is available but is not currently processing any streams •...

  • Page 25: Http Status Reporting For Publisher And Replicator

    the cause is unknown or a configuration or connectivity issue has been resolved, administrators may wish to consider Republishing the subscription. • Last Connection: the amount of time elapsed since the subscriber queried the Publisher for stream events • Last Transmit: the amount of time elapsed since the subscriber last retrieved at least one stream event from the Publisher •...

  • Page 26: Replicator Response

    [enumerator1] uuid=... channel={rules file channel for which the enumerator was instantiated} context={optional description for enumerator} host=... version=... status=... eventsMatched=... backlogTotal=... transmitQueue=... subscriberQueue=... subscriberInProgress=... droppedEvents=... lastConnection=... lastTransmit=... [enumerator2] uuid=... [enumerator3] uuid=... 5.5.2. Replicator Response [settings] hostip={replicator ip address or host name} targetCluster={target cluster multicast group} [stats] version={replicator software version}...

  • Page 27: Request For Source Cluster Ip Addresses

    uuid=... 5.5.3. Request for Source Cluster IP Addresses Publisher clients such as Replicator may need to know IP addresses of nodes in the CAStor source cluster that Publisher listens to (in order to make SCSP requests to this cluster). The Publisher Channel Server will support a request for a list of source cluster IP addresses and SCSP port numbers: GET /sourceclusteripports.bin HTTP/1.1...

  • Page 28: Support For Content Restoration And Fail-Over

    Chapter 6. Support for Content Restoration and Fail- Over Some common uses of DX Content Router are disaster recovery, backup, and archiving. In all these cases, it will sometimes be necessary to recover lost content data from a remote cluster. The exact nature of such a recovery process, and the ease and speed with which it happens, will depend somewhat on the topology of the cluster network.

  • Page 29: Content Metadata

    Appendix A. Content Metadata In planning for replication of data from one location to another and/or disaster recovery scenarios within DX Storage, it is advisable to ensure that you are both fully utilizing the system metadata automatically stored with every stream as well as adding custom metadata that will allow you to create dynamic rules for distributing your content.

  • Page 30: Dx Content Router Configuration

    Appendix B. DX Content Router Configuration In order for DX Content Router to function correctly, each installed service has a separate configuration file. Best practice is to copy the sample config files located in the /etc/caringo/ contentrouter directory with the name of each service: $ sudo su - cd /etc/caringo/contentrouter cp publisher.cfg.sample publisher.cfg...
  • Page 31 Option Name Default Mandatory Description like logrotate is preferred. loglevel The level of logging verbosity with the following supported values: 50=critical, 40=error, 30=warning, 20=info, 10=debug, 0=no log group 225.0.10.100 The multicast group address for the DX Storage cluster from which UUIDs are gathered.
  • Page 32 Option Name Default Mandatory Description subscribers access to the DX Storage cluster for replication rulesFile /etc/caringo/ The name and contentrouter/ location of the XML rules.xml rules file the Publisher uses to filter UUIDs consolePort 8090 The port for the Publisher web console publicationServerPort The port Publisher uses to publish UUID...
  • Page 33 Option Name Default Mandatory Description that do not send an errOfflineAfter parameter at runtime. subscriberTimeout 90000 Time in seconds Interval before the Publisher terminates a Subscriber if it has not been heard from. Applies only to Subscribers that do not send a Timeout parameter at runtime.

  • Page 34: Replicator Configuration

    Option Name Default Mandatory Description query argument on a Start or Next request. Replicator throughput can be increased by increasing the value of this parameter. publicationServerStrict False Determines whether ArgsChecking Enumerator API query argument syntax is strictly enforced on requests to the Publisher.
  • Page 35 Option Name Default Mandatory Description if local file-based logging will be used instead of syslog. loglevel The level of logging verbosity with the following supported values: 50=critical, 40=error, 30=warning, 20=info, 10=debug, 0=no log logbackups The number of older, rotated log files to keep when file-based logging is used.
  • Page 36 Option Name Default Mandatory Description boot. Either scsphosts or cluster must be specified. castorProxyIp none IP address of a proxy configured in front of the target cluster castorProxyPort none Port of a proxy configured in front of the target cluster subscribeTo none Specifies one or more...
  • Page 37 Option Name Default Mandatory Description state on this port. There is not currently a separate console for Replicator. storageDir /var/opt/caringo/ A unique, writable contentrouter/ directory path for use replicator in storing in process stream information. maxActiveEvents The number of events Replicator should process at one time.

  • Page 38: Enumerator Api

    Appendix C. Enumerator API DX Content Router provides a public HTTP 1.1 server interface component as part of the DX Content Router Publisher service. The purpose of the HTTP interface is to enable a simple standards based approach for building "plug-in like" DX Storage internal and 3rd party applications that require some form of object enumeration.

  • Page 39: Enumerator Start Query Arguments

    POST /?type=&start=&end= HTTP/1.1 Host: Here channel is the "subscription name" as specified when configuring a DX Content Router replicator service. It corresponds to one of the sets of filter rules identified by a select tag in the publisher rules.xml file. The start and end parameters delimit the create dates of objects to be enumerated for metadata and UUID enumerator types.

  • Page 40: Enumerator Next

    C.3. Enumerator Next The Enumerator Next command is a request for the next objects in an enumeration which was previously initiated with an Enumerator Start command. For performance reasons Publisher does not attempt to retrieve elements in a specific order. The format of the request is as follows: GET /?maxItems=...

  • Page 41: Enumerator Next Response

    C.3.2. Enumerator Next Response A typical normal response to an Enumerator Next command of type UUID or Event is: HTTP/1.1 200 Date: ... Server: Content Router Publisher Content-Type: text/plain Content-Length: Content-Sync-Token: For a Metadata Enumerator, the response has an additional "Content-UUID" header containing the object's UUID.

  • Page 42: Enumerator Timeout

    If for any reason the request is not successful, a 404 response code will be returned with a descriptive message in the response body as to the encountered problem. C.5. Enumerator Timeout The DX Content Router Publisher will repurpose the existing configuration parameter called subscriberTimeout.

  • Page 43: Version 2.2 December

    Argument Name Description expected interval between enumerator queries. When not specified, a default value for errOfflineAfter will be derived from the Publisher's "subscriberErrOfflineAfter" configuration value. timeout Number of seconds before the Publisher will terminate an enumerator if it has not been heard from.

This manual is also suitable for:

Dx6004sDx6012sDx content router

Table of Contents