MapReduce Service (MRS)¶

MapReduce Service (MRS) provides enterprise-level big data clusters on the cloud. Tenants can fully control the clusters and easily run big data components such as Hadoop, Spark, HBase, Kafka, and Storm in the clusters.

Table of Contents

Cluster ¶

MRS cluster management.

List MRS Clusters ¶

This interface is used to query a list of clusters created by a user.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

for cluster in conn.mrs.clusters():
    print(cluster)

Get MRS Cluster ¶

This interface is used to get a MRS Cluster by ID or an instance of class ClusterInfo.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

cluster = 'cluster_id'
cluster = conn.mrs.get_cluster(cluster=cluster)
print(cluster)

Find MRS Cluster ¶

This interface is used to find a MRS cluster by name or ID. The return value is a instance of class ClusterInfo.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
cluster = conn.mrs.find_cluster(name_or_id)
print(cluster)

Update MRS Cluster ¶

This interface is used to update a MRS Cluster instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')


attrs = {
    'parameters': {
        'scale_type': 'scale_out',
        'node_id': 'node_orderadd',
        'instances': "1",
        'node_group': 'task_node_default_group'
    }
}

cluster = 'clstr1'
cluster = conn.mrs.find_cluster(name_or_id=cluster)
cluster = conn.mrs.update_cluster(cluster=cluster, **attrs)
print(cluster)

Delete MRS Cluster ¶

This interface is used to delete a MRS Cluster instance by id or an instance of class Cluster.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
cluster = conn.mrs.find_cluster(name_or_id)
conn.mrs.delete_cluster(cluster=cluster)

List MRS Cluster Hosts ¶

This interface is used to query a list of cluster hosts created by a user.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

for host in conn.mrs.hosts():
    print(host)

Datasources ¶

List MRS Datasources ¶

This interface is used to query MRS Datasources and to filter the output with query parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

for datasource in conn.mrs.datasources():
    print(datasource)

Get MRS Datasource ¶

This interface is used to get a MRS Datasource by ID or an instance of class Datasource.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

datasource = 'ds_id'
datasource = conn.mrs.get_cluster(datasource=datasource)
print(datasource)

Create MRS Datasource ¶

This interface is used to create a MRS Datasource instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

attrs = {
    'name': 'my-data-source',
    'url': '/simple/mapreduce/input',
    'is_protected': False,
    'is_public': False,
    'type': 'hdfs',
    'description': 'this is the data source template'
}

datasource = conn.mrs.create_datasource(**attrs)
print(datasource)

Update MRS Datasource ¶

This interface is used to update a MRS Datasource instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')


attrs = {
    "name": "my-data-source-update",
    "url": "/simple/mapreduce/input",
    "is_protected": False,
    "is_public": False,
    "type": "hdfs",
    "description": "this is the data source template"
}

datasource = 'ds1'
datasource = conn.mrs.find_datasource(name_or_id=datasource)
datasource = conn.mrs.update_datasource(datasource=datasource, **attrs)
print(datasource)

Delete MRS Datasource ¶

This interface is used to delete a MRS Datasource instance by id or an instance of class Datasource.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
datasource = conn.mrs.find_datasource(name_or_id)
conn.mrs.delete_datasource(datasource=datasource)

Find MRS Datasource ¶

This interface is used to find a MRS Datasource by name or ID. The return value is a instance of class Datasource.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
datasource = conn.mrs.find_datasource(name_or_id)
print(datasource)

Job binaries ¶

List MRS Job binaries ¶

This interface is used to query MRS Job binaries and to filter the output with query parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

for job_binary in conn.mrs.jobbinaries():
    print(job_binary)

Get MRS Job binary ¶

This interface is used to get a MRS Job binary by ID or an instance of class Jobbinary.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

job_binary = 'job_binary_id'
job_binary = conn.mrs.get_jobbinary(jobbinary=job_binary)
print(job_binary)

Create MRS Job binary ¶

This interface is used to create a MRS Job binary instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

attrs = {
    "name": "my-job-binary",
    "url": "/simple/mapreduce/program",
    "is_protected": False,
    "is_public": False,
    "description": "this is the job binary template"
}

job_binary = conn.mrs.create_jobbinary(**attrs)
print(job_binary)

Update MRS Job binary ¶

This interface is used to update a MRS Job binary instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')


attrs = {
    "name": "my-job-binary-update",
    "url": "/simple/mapreduce/program",
    "is_protected": False,
    "is_public": False,
    "description": "this is the job binary template"
}

job_binary = 'jb1'

Delete MRS Job binary ¶

This interface is used to delete a MRS Job binary instance by id or an instance of class Jobbinary.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
job_binary = conn.mrs.find_jobbinary(name_or_id)
conn.mrs.delete_jobbinary(jobbinary=job_binary)

Find MRS Job binary ¶

This interface is used to find a MRS Job binary by name or ID. The return value is a instance of class Jobbinary.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
job_binary = conn.mrs.find_jobbinary(name_or_id)
print(job_binary)

Job ¶

List MRS Job ¶

This interface is used to query MRS Job and to filter the output with query parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

for job in conn.mrs.jobs():
    print(job)

Get MRS Job ¶

This interface is used to get a MRS Job by ID or an instance of class Job.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

job = 'job_id'
job = conn.mrs.get_job(job=job)
print(job)

Create MRS Job ¶

This interface is used to create a MRS Job instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

attrs = {
    "name": "my-mapreduce-job",
    "mains": [],
    "libs": [
        "092b628b-26a3-4571-9ba4-f8d000df8877"
    ],
    "is_protected": False,
    "interface": [],
    "is_public": False,
    "type": "MapReduce",
    "description": "This is the Map Reduce job template"
}

job = conn.mrs.create_job(**attrs)
print(job)

Update MRS Job ¶

This interface is used to update a MRS Job instance with parameters.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

attrs = {
    "name": "my-mapreduce-job-update",
    "mains": [],
    "libs": [
        "2628d0e4-6109-4a09-a338-c4ee1b0963ed"
    ],
    "is_protected": False,
    "interface": [],
    "is_public": False,
    "type": "MapReduce",
    "description": "This is the Map Reduce job template"
}

job = 'job1'
job = conn.mrs.find_job(name_or_id=job)
job = conn.mrs.update_job(job=job, **attrs)
print(job)

Delete MRS Job ¶

This interface is used to delete a MRS Job instance by id or an instance of class Job.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
job = conn.mrs.find_job(name_or_id)
conn.mrs.delete_job(job=job)

Find MRS Job ¶

This interface is used to find a MRS Job by name or ID. The return value is a instance of class Job.

import openstack

openstack.enable_logging(True)
conn = openstack.connect(cloud='otc')

name_or_id = 'name_or_id'
job = conn.mrs.find_job(name_or_id)
print(job)

last updated: 2023-08-18 10:18