Friday, 2 August 2019

CQL data types

CQL data types

CQL Type
Constants supported
Description
ascii
strings
US-ASCII character string
bigint
integers
64-bit signed long
blob
blobs
Arbitrary bytes (no validation), expressed as hexadecimal
boolean
booleans
true or false
counter
integers
Distributed counter value (64-bit long)
date
strings
Value is a date with no corresponding time value; Cassandra encodes date as a 32-bit integer representing days since epoch (January 1, 1970). Dates can be represented in queries and inserts as a string, such as 2015-05-03 (yyyy-mm-dd)
decimal
integers, floats
Variable-precision decimal
Java type
Note: When dealing with currency, it is a best practice to have a currency class that serializes to and from an int or use the Decimal form.
double
integers, floats
64-bit IEEE-754 floating point
Java type
float
integers, floats
32-bit IEEE-754 floating point
Java type
frozen
user-defined types, collections, tuples
A frozen value serializes multiple components into a single value. Non-frozen types allow updates to individual fields. Cassandra treats the value of a frozen type as a blob. The entire value must be overwritten.
Note: Cassandra no longer requires the use of frozen for tuples:
frozen <tuple <int, tuple<text, double>>>
inet
strings
IP address string in IPv4 or IPv6 format, used by the python-cql driver and CQL native protocols
int
integers
32-bit signed integer
list
n/a
A collection of one or more ordered elements: [literal, literal, literal].
CAUTION:
Lists have limitations and specific performance considerations. Use a frozen list to decrease impact. In general, use a set instead of list.
map
n/a
A JSON-style array of literals: { literal : literal, literal : literal ... }
set
n/a
A collection of one or more elements: { literal, literal, literal }
smallint
integers
2 byte integer
text
strings
UTF-8 encoded string
time
strings
Value is encoded as a 64-bit signed integer representing the number of nanoseconds since midnight. Values can be represented as strings, such as 13:30:54.234.
timestamp
integers, strings
Date and time with millisecond precision, encoded as 8 bytes since epoch. Can be represented as a string, such as 2015-05-03 13:30:54.234.
timeuuid
uuids
Version 1 UUID only
tinyint
integers
1 byte integer
tuple
n/a
A group of 2-3 fields.
uuid
uuids
A UUID in standard UUID format
varchar
strings
UTF-8 encoded string
varint
integers
Arbitrary-precision integer
Java type

Blob type
The Cassandra blob data type represents a constant hexadecimal number defined as 0[xX](hex)+ where hex is a hexadecimal character, such as [0-9a-fA-F]. For example, 0xcafe. The maximum theoretical size for a blob is 2 GB. The practical limit on blob size, however, is less than 1 MB. A blob type is suitable for storing a small image or short string.
Blob conversion functions 
These functions convert the native types into binary data (blob):
typeAsBlob(value)
blobAsType(value)
For every native, nonblob data type supported by CQL, the typeAsBlob function takes a argument of that data type and returns it as a blob. Conversely, the blobAsType function takes a 64-bit blob argument and converts it to a value of the specified data type, if possible.
This example shows how to use bigintAsBlob:
CREATE TABLE bios ( user_name varchar PRIMARY KEY,    bio blob );
INSERT INTO bios (user_name, bio) VALUES ('fred', bigintAsBlob(3));
SELECT * FROM bios;
user_name | bio
---------------+--------------------
fred            | 0x0000000000000003

Collection type
A collection column is declared using the collection type, followed by another type, such as int or text, in angle brackets. For example, you can create a table having a list of textual elements, a list of integers, or a list of some other element types.
list<text>
list<int>
Collection types cannot be nested, but frozen collection types can be nested inside frozen or non-frozen collections. For example, you may define a list within a list, provided the inner list is frozen:
list<frozen <list<int>>>
Indexes may be created on a collection column of any type.
Using frozen in a collection 
A frozen value serializes multiple components into a single value. Non-frozen types allow updates to individual fields. Cassandra treats the value of a frozen type as a blob. The entire value must be overwritten.
column_name collection_type<data_type, frozen<column_name>>
For example:
CREATE TABLE mykeyspace.users (
  id uuid PRIMARY KEY,
  name frozen <fullname>,
  direct_reports set<frozen <fullname>>,     // a collection set
  addresses map<text, frozen <address>>     // a collection map
  score set<frozen <set<int>>>              // a set with a nested frozen set
);

UUID and timeuuid types
The UUID (universally unique id) comparator type is used to avoid collisions in column names. Alternatively, you can use the timeuuid.
Timeuuid types can be entered as integers for CQL input. A value of the timeuuid type is a Type 1 UUID. A Version 1 UUID includes the time of its generation and are sorted by timestamp, making them ideal for use in applications requiring conflict-free timestamps. For example, you can use this type to identify a column (such as a blog entry) by its timestamp and allow multiple clients to write to the same partition key simultaneously. Collisions that would potentially overwrite data that was not intended to be overwritten cannot occur.
UUID and timeuuid types

Using a counter
To load data into a counter column, or to increase or decrease the value of the counter, use the UPDATE command. Cassandra rejects USING TIMESTAMP or USING TTL in the command to update a counter column.
Procedure
Create a table for the counter column.
cqlsh> USE cycling;
cqlsh> CREATE TABLE popular_count (id UUID PRIMARY KEY,  popularity counter);
Loading data into a counter column is different than other tables. The data is updated rather than inserted.
UPDATE cycling.popular_count SET popularity = popularity + 1  WHERE id = 6ab09bec-e68e-48d9-a5f8-97e6fb4c9b47;
Take a look at the counter value and note that popularity has a value of 1.
SELECT * FROM cycling.popular_count;

Additional increments or decrements will change the value of the counter column.

UUID and timeuuid functions
The uuid() function takes no parameters and generates a random Type 4 UUID suitable for use in INSERT or UPDATE statements.
Several timeuuid() functions are designed for use with the timeuuid() type:
dateOf()
Used in a SELECT clause, this function extracts the timestamp of a timeuuid column in a result set. This function returns the extracted timestamp as a date. Use unixTimestampOf() to get a raw timestamp.
now()
In the coordinator node, generates a new unique timeuuid in milliseconds when the statement is executed. The timestamp portion of the timeuuid conforms to the UTC (Universal Time) standard. This method is useful for inserting values. The value returned by now() is guaranteed to be unique.
minTimeuuid() and maxTimeuuid()
Returns a UUID-like result given a conditional time component as an argument. For example:
SELECT * FROM myTable WHERE t > maxTimeuuid('2013-01-01 00:05+0000') AND t < minTimeuuid('2013-02-02 10:00+0000')
The min/maxTimeuuid example selects all rows where the timeuuid column, t, is strictly later than 2013-01-01 00:05+0000 but strictly earlier than 2013-02-02 10:00+0000. The t >= maxTimeuuid('2013-01-01 00:05+0000') does not select a timeuuid generated exactly at 2013-01-01 00:05+0000 and is essentially equivalent to t > maxTimeuuid('2013-01-01 00:05+0000').
The values returned by minTimeuuid and maxTimeuuid functions are not true UUIDs in that the values do not conform to the Time-Based UUID generation process specified by the RFC 4122. The results of these functions are deterministic, unlike the now() function.
unixTimestampOf()
Used in a SELECT clause, this functions extracts the timestamp in milliseconds of a timeuuid column in a result set. Returns the value as a raw, 64-bit integer timestamp.
Cassandra 2.2 and later support some additional timeuuid and timestamp functions to manipulate dates. The functions can be used in INSERT, UPDATE, and SELECT statements.
toDate(timeuuid)
Converts timeuuid to date in YYYY-MM-DD format.
toTimestamp(timeuuid)
Converts timeuuid to timestamp format.
toUnixTimestamp(timeuuid)
Converts timeuuid to UNIX timestamp format.
toDate(timestamp)
Converts timestamp to date in YYYY-MM-DD format.
toUnixTimestamp(timestamp)
Converts timestamp to UNIX timestamp format.
toTimestamp(date)
Converts date to timestamp format.
toUnixTimestamp(date)
Converts date to UNIX timestamp format.
An example of the new functions creates a table and inserts various time-related values:
CREATE TABLE sample_times (a int, b timestamp, c timeuuid, d bigint, PRIMARY KEY (a,b,c,d));
INSERT INTO sample_times (a,b,c,d) VALUES (1, toUnixTimestamp(now()), 50554d6e-29bb-11e5-b345-feff819cdc9f, toTimestamp(now()));
Select data and convert it to a new format:

SELECT toDate(c) FROM sample_times;

Friday, 19 July 2019

Operations on Keyspace in Cassandra

Cassandra's Keyspace operations

Create Keyspace
A keyspace is an object that holds the column families, user defined types. In Cassandra, Keyspace is similar to RDBMS Database. Keyspace holds column families, indexes, user defined types, data center awareness, strategy used in keyspace, replication factor, etc.
Command "Create Keyspace" is used to create keyspace in Cassandra.
Syntax
Create keyspace KeyspaceName with replicaton={'class':strategy name, 'replication_factor': No of replications on different nodes};
Various Components of Cassandra Keyspace
Strategy: While declaring strategy name in Cassandra. There are two kinds of strategies declared in Cassandra Syntax.
1.Simple Strategy: Simple strategy is used when you have just one data center. In this strategy, the first replica is placed on the node selected by the partitioner. Remaining nodes are placed in the clockwise direction in the ring without considering rack or node location.
2. Network Topology Strategy: Network topology strategy is used when you have more than one data centers. In this strategy, you have to provide replication factor for each data center separately. Network topology strategy places replicas in nodes in the clockwise direction in the same data center. This strategy attempts to place replicas in different racks.
Replication Factor: Replication factor is the number of replicas of data placed on different nodes. For no failure, 3 is good replication factor. More than two replication factor ensures no single point of failure. Sometimes, the server can be down, or network problem can occur, then other replicas provide service with no failure.
Example: Here is the snapshot of the executed command "Create Keyspace" that will create keyspace in Cassandra.
cqlsh> create keyspace University with replication = {'class':'SimpleStrategy', replication_factor':3};
Details of above query : 
Command to create keyspace(create keyspace)
Keyspace Name (University)
Strategy Name (SimpleStrategy)

Replication Factor('3')
After successful execution of command "Create Keyspace", Keyspace University will be created in Cassandra with strategy "SimpleStrategy" and replication factor 3.

Alter Keyspace
Command "Alter Keyspace" alters the replication factor, strategy name and durable writes properties in created keyspace in Cassandra.
Syntax
Alter Keyspace KeyspaceName with replication={'class':'StrategyName', 'replication_factor': no of replications on different nodes} with DURABLE_WRITES=true/false
Key aspects while altering Keyspace in Cassandra
Keyspace Name: Keyspace name cannot be altered in Cassandra.
Strategy Name: Strategy name can be altered by specifying new strategy name.
Replication Factor: Replication factor can be altered by specifying new replication factor.
DURABLE_WRITES :DURABLE_WRITES value can be altered by specifying its value true/false. By default, it is true. If set to false, no updates will be written to the commit log and vice versa.

Example:
cqlsh>Alter Keyspace University with replication={'class':'NetworktopologyStrategy', 'DataCenter1':1};
Details of above query : 
Command to alter keyspace : Alter keyspace
Replication factor changed from 3 to 1 for DataCenter1 : 'DataCenter1':1

Strategy name changed from SimpleStrategy to NetworktopologyStrategy
After successful execution of the command "Alter Keyspace", Strategyname will be changed from 'SimpleStrategy' to 'NetworkTopologyStrategy' and replication factor will be changed from 3 to 1 for 'DataCenter1.'

Drop/Delete Keyspace

Command 'Drop Keyspace' drops keyspace including all the data, column families, user defined types and indexes from Cassandra. Before dropping the keyspace, Cassandra takes a snapshot of the keyspace. If keyspace does not exist in the Cassandra, Cassandra will return an error unless IF EXISTS is used.
Syntax
Drop keyspace KeyspaceName
OR
DROP keyspace [IF EXISTS] KeyspaceName
Example

cqlsh>Drop keyspace University;
After successful execution of the command 'Drop keyspace University', keyspace University will be dropped from Cassandra with all the data and schema.
Below error is returned when tried to access keyspace that does not exist.
cqlsh>use University;
InvalidRequest: code=2200 [Invalid query] message="Keyspace 'university' does not exist"

Note: There is no difference in drop keyspace and delete keyspace. Drop keyspace is equal to delete keyspace.

Wednesday, 17 July 2019

Cassandra Architecture sub-point

Calculating Tokens for a Multi-Data Center Cluster
In multi-data center deployments, replica placement is calculated per data center using the NetworkTopologyStrategy replica placement strategy. In each data center (or replication group) the first replica for a particular row is determined by the token value assigned to a node. Additional replicas in the same data center are placed by walking the ring clockwise until it reaches the first node in another rack.
If you do not calculate partition-er tokens so that the data ranges are evenly distributed for each data center, you could end up with uneven data distribution within a data center. The goal is to ensure that the nodes for each data center are evenly dispersed around the ring, or to calculate tokens for each replication group individually (without conflicting token assignments).

One way to avoid uneven distribution is to calculate tokens for all nodes in the cluster, and then alternate the token assignments so that the nodes for each data center are evenly dispersed around the ring.
Another way to assign tokens in a multi data center cluster is to generate tokens for the nodes in one data center, and then offset those token numbers by 1 for all nodes in the next data center, by 2 for the nodes in the next data center, and so on. This approach is good if you are adding a data center to an established cluster, or if your data centers do not have the same number of nodes.
strategy_options
Specifies configuration options for the chosen replication strategy.
For SimpleStrategy, it specifies replication_factor in the format of replication_factor:number_of_replicas.
For NetworkTopologyStrategy, it specifies the number of replicas per data center in a comma separated list of datacenter_name:number_of_replicas. Note that what you specify for datacenter_name depends on the cluster-configured snitch you are using. There is a correlation between the data center name defined in the keyspace strategy_options and the data center name as recognized by the snitch you are using. The nodetool ring command prints out data center names and rack locations of your nodes if you are not sure what they are.
See Choosing Keyspace Replication Options for guidance on how to best configure replication strategy and strategy options for your cluster.
Setting and updating strategy options with the Cassandra CLI requires a slightly different command syntax than other attributes; note the brackets and curly braces in this example:

[default@unknown] CREATE KEYSPACE test WITH placement_strategy = 'NetworkTopologyStrategy' AND strategy_options=[{us-east:6,us-west:3}];

Choosing Keyspace Replication Options
When you create a keyspace, you must define the replica placement strategy and the number of replicas you want.
DataStax recommends always choosing NetworkTopologyStrategy for both single and multi-data center clusters. It is as easy to use as SimpleStrategy and allows for expansion to multiple data centers in the future, should that become useful. It is much easier to configure the most flexible replication strategy up front, than to reconfigure replication after you have already loaded data into your cluster.
NetworkTopologyStrategy takes as options the number of replicas you want per data center. Even for single data center (or single node) clusters, you can use this replica placement strategy and just define the number of replicas for one data center. For example (using cassandra-cli):
[default@unknown] CREATE KEYSPACE test WITH placement_strategy = 'NetworkTopologyStrategy' AND strategy_options=[{us-east:6}];
Or for a multi-data center cluster:
[default@unknown] CREATE KEYSPACE test WITH placement_strategy = 'NetworkTopologyStrategy' AND strategy_options=[{DC1:6,DC2:6,DC3:3}];
When declaring the keyspace strategy_options, what you name your data centers depends on the snitch you have chosen for your cluster. The data center names must correlate to the snitch you are using in order for replicas to be placed in the correct location.
As a general rule, the number of replicas should not exceed the number of nodes in a replication group. However, it is possible to increase the number of replicas, and then add the desired number of nodes afterwards. When the replication factor exceeds the number of nodes, writes will be rejected, but reads will still be served as long as the desired consistency level can be met.

listen_address

The IP address or hostname that other Cassandra nodes will use to connect to this node. If left blank, you must have hostname resolution correctly configured on all nodes in your cluster so that the hostname resolves to the correct IP address for this node (using /etc/hostname, /etc/hosts or DNS).

Configuring the PropertyFileSnitch
The PropertyFileSnitch requires you to define network details for each node in the cluster in a cassandra-topology.properties configuration file. A sample of this file is located in /etc/cassandra/conf/cassandra.yaml in packaged installations or $CASSANDRA_HOME/conf/cassandra.yaml in binary installations.
Every node in the cluster should be described in this file, and this file should be exactly the same on every node in the cluster if you are using the PropertyFileSnitch.

Architecture of Cassandra

Architecture of Cassandra A Cassandra instance is a collection of independent nodes that are configured together into a cluster. In a C...