Services and State with Ambari REST API

In this post we are going to look a little closer at the way the Ambari API can be used to manage Hadoop services. At the end of this post you will find a list of all the currently supported Hadoop services with all the needed master, slave and client components that can be manged and administrated within your HDP stack. Also this posts contains the possible states and state transitions a component might have which could become useful when facing problems like Host config is in invalid state.

The possible states a service component can be in is listed below. To change or get the state of a service component the following API calls can be issued.

<code>// Getting the state of the component</code>

<code>curl -k -u admin:admin -H </code><code>"X-Requested-By:ambari"</code> <code>-i -X GET</code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/hosts/&lt;HOST_FQDN&gt;/host_components/YARN_CLIENT </code>

<code>// Setting the state of the component to INSTALLED</code>

<code>curl -k -u admin:admin -H </code><code>"X-Requested-By:ambari"</code> <code>-i -X PUT </code>

<code>-d '{</code><code>"RequestInfo"</code><code>:{</code><code>"context"</code><code>:</code><code>"Install YARN_CLIENT"</code><code>},</code>

<code>"Body"</code><code>:{</code><code>"HostRoles"</code><code>:{</code><code>"state"</code><code>:</code><code>"INSTALLED"</code><code>}}}' </code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/hosts/&lt;HOST_FQDN&gt;/host_components/YARN_CLIENT</code>

Among the need to change the state of a service component a very common issue is the need to actually move one service – most likely a master service – to different host. Users faced with this problems often need to issue the existing API due to a lack of support by the existing UI – at least with older releases of Ambari Web UI. This is an example of moving the OOZIE_SERVER to a new host.

<code>// Stop the old component by putting it back in state INSTALLED</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-By:ambari"</code> <code>-i -X PUT </code>

<code>-d </code><code>'{"RequestInfo": {"context": "Stop Oozie","query":"HostRoles/component_name.in('</code><code>OOZIE_SERVER')"}, </code>

<code>"Body"</code><code>:{</code><code>"HostRoles"</code><code>: {</code><code>"state"</code><code>: </code><code>"INSTALLED"</code><code>}}}' </code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/hosts</code>

<code>/&lt;OLD_HOST_FQDN&gt;/host_components/OOZIE_SERVER</code>

<code>// Add component to host</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-By:ambari"</code> <code>-i -X POST -d </code>

<code>/&lt;NEW_HOST_FQDN&gt;/host_components/OOZIE_SERVER</code>

<code>// Install component by achieving stated INSTALLED</code>

<code>-d </code><code>'{"RequestInfo": {"context": "Install Oozie","query":"HostRoles/component_name.in('</code><code>OOZIE_SERVER')"}, </code>

<code>// Delete old component</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-By:ambari"</code> <code>-i -X DELETE </code>

<code>// Starting the new component</code>

<code>-d </code><code>'{"RequestInfo": {"context": "Start Oozie","query":"HostRoles/component_name.in('</code><code>OOZIE_SERVER')"}, </code>

<code>"Body"</code><code>:{</code><code>"HostRoles"</code><code>: {</code><code>"state"</code><code>: </code><code>"STARTED"</code><code>}}}' </code>

<code>// Get service info</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-by:ambari"</code> <code>-i -k -X GET</code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/services/KNOX/</code>

<code>// Stop component KNOX_GATEWAY</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-by:ambari"</code> <code>-i -k -X PUT </code>

<code>-d </code><code>'{"ServiceComponentInfo": {"state": "INSTALLED"}}'</code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/services/KNOX/components/KNOX_GATEWAY</code>

<code>// Stop service KNOX</code>

<code>-d </code><code>'{"ServiceInfo": {"state": "INSTALLED"}}'</code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/services/KNOX</code>

<code>// Delete component</code>

<code>curl -u admin:admin -H </code><code>"X-Requested-by:ambari"</code> <code>-i -k -X DELETE</code>

<code>http://&lt;HOST&gt;:8080/api/v1/clusters/&lt;CLUSTER_NAME&gt;/hosts/&lt;HOST_FQDN&gt;/host_components/KNOX_GATEWAY</code>

<code>// Delete service</code>

Possible states of a component:

StateDescription

INITInitial/Clean state.

INSTALLINGIn the process of installing.

INSTALL_FAILEDInstall failed.

INSTALLEDState when install completed successfully.

STARTINGIn the process of starting.

STARTEDState when start completed successfully.

STOPPINGIn the process of stopping.

UNINSTALLINGIn the process of uninstalling.

UNINSTALLEDState when uninstall completed successfully.

WIPING_OUTIn the process of wiping out the install.

UPGRADINGIn the process of upgrading the deployed bits.

DISABLEDDisabled master’s backup state.

UNKNOWNState could not be determined.

Allowed State transitions of a component:

Desired StateStart State

INSTALLED &lt;-INIT, UNINSTALLED, INSTALLED, INSTALLING, STARTED, INSTALL_FAILED, UPGRADING, STOPPING, UNKNOWN, DISABLED

STARTED &lt;-INSTALLED, STARTING, STARTED

UNINSTALLED &lt;-INSTALLED, UNINSTALLED, UNINSTALLING

INIT &lt;-UNINSTALLED, INIT, WIPING_OUT

DISABLED &lt;-INSTALLED, INSTALL_FAILED, UNKNOWN

A Hadoop cluster contains of multiple services that individually have either master, slave or client components. Below you will find a list of all currently supported components as part of a HDP stack divided in master, slave, or client groups. The cardinality notates the number of components that can exist at the same time in one cluster.

NameAmbari Component Name

Service

Cardinality

NameNodeNAMENODEHDFS1-2

Secondary NameNodeSECONDARY_NAMENODEHDFS1

ResourceMangerRESOURCEMANAGERYARN1-2

Application Timeline ServerAPP_TIMELINE_SERVERYARN1

HistoryServerHISTORYSERVERMAPREDUCE2 1

Hive MetastoreHIVE_METASTOREHIVE1-2

HiveServer2HIVE_SERVERHIVE1-2

WebHcat ServerWEBHCAT_SERVERHIVE1

HBase MasterHBASE_MASTERHBASE1+

Spark Job History ServerSPARK_JOBHISTORYSERVERSPARK1

Nimbus ServerNIMBUSSTORM1

Storm REST ServerSTORM_REST_APISTORM1

Storm UISTORM_UI_SERVERSTORM1

DRPC ServerDRPC_SERVERSTORM1

Falcon ServerFALCON_SERVERFALCON1

ZookeeperZOOKEEPER_SERVERZOOKEEPER1+ (odd #)

Kafka BrokerKAFKA_BROKERKAFKA1+

Knox GatewayKNOX_GATEWAYKNOX1+

Ranger Admin ServerRANGER_ADMINRANGER1-3

Ranger User SyncRANGER_USERSYNCRANGER1

Ranger Key Management ServerRANGER_KMS_SERVERRANGER_KMS1+

Oozie ServerOOZIE_SERVEROOZIE1

Ganglia ServerGANGLIA_SERVERGANGLIA1

Nagios ServerNAGIOS_SERVERNAGIOS1

Ambari Metrics ServiceMETRICS_COLLECTORAMS1

Zeppelin ServerZEPPELIN_MASTERSPARK / HIVE1

ServiceCardinality

DataNodeDATANODEHDFS1+

Journale Nodes for NameNode HAJOURNALNODEHDFS0+ (odd #)

Zookeeper Failover ServiceZKFCHDFS0+

Secondary NameNodeNFS_GATEWAYHDFS0+

Node ManagerNODEMANAGERYARN1+

HBase RegionServerHBASE_REGIONSERVERHBASE1+

Phoneix Query ServerPHOENIX_QUERY_SERVERHBASE0+

Storm SupervisorSUPERVISORSTORM1+

Ganglia Metrics CollectorGANGLIA_MONITORGANGLIAALL

Ambari Metrics CollectorMETRICS_MONITORAMSALL

HDFS ClientHDFS_CLIENTHDFS1+

YARN ClientYARN_CLIENTYARN1+

MapReduce ClientMAPREDUCE2_CLIENTMAPREDUCE21+

Spark ClientSPARK_CLIENTSPARK1+

Falcon ClientFALCON_CLIENTFALCON1+

HBase ClientHBASE_CLIENTHBASE1+

Hive ClientHIVE_CLIENTHIVE1+

HCat ClientHCATHIVE1+

Mahout ClientMAHOUTMAHOUT0+

Oozie ClientOOZIE_CLIENTOOZIE1+

Sqoop ClientSQOOPSQOOP1+

Zookeeper ClientZOOKEEPER_CLIENTZOOKEEPER1+

本文轉自 yntmdr 51CTO部落格，原文連結：http://blog.51cto.com/yntmdr/1775098，如需轉載請自行聯系原作者