Remove beeline cli hack from hadoop (2059bc29) · Commits · Services / Code Samples

computing/hadoop/HadoopShell.ipynb

+0 −133

Original line number	Diff line number	Diff line
		%% Cell type:markdown id:4e4a4556-96f1-4bf4-91d0-8cd98526c53e tags:

		# Using the Hadoop cluster

		This notebook contains instruction on how to use the Hadoop cluster from the command line

		# Setup

		## Authentication

		The Hadoop cluster uses Kerberos for authentication. You need to initialize your credentials with Kerberos to be able to connect to the Hadoop platform.

		### Initializing credentials

		You have to run the `kinit` command in a regular terminal.

		This can't be run in a notebook because it will ask for your password.

		You can check the help with

		%% Cell type:code id:3f64a948-9232-4122-867c-6a398ab48fea tags:

		``` python
		!man kinit
		```

		%% Cell type:markdown id:0f6dcdea-7157-4eeb-8108-130c83e44276 tags:

		### Listing active credentials

		%% Cell type:code id:14539960-6005-4ae5-b883-e0a35ba11629 tags:

		``` python
		!klist
		```

		%% Output

		Ticket cache: FILE:/tmp/krb5cc_63532
		Default principal: torradeflot@PIC.ES

		Valid starting Expires Service principal
		11/25/24 12:49:18 11/26/24 12:49:12 krbtgt/PIC.ES@PIC.ES

		%% Cell type:markdown id:fad17362-fa12-4320-8cd2-1a354c90f407 tags:

		### Destroy credentials

		%% Cell type:code id:f75fbe03-60f5-449c-ba22-0a7bdad92597 tags:

		``` python
		!kdestroy
		```

		%% Cell type:markdown id:779dcd3a-408f-4735-820c-03927948aaf6 tags:

		## Environment modification

		At the time of writing this documentation there are some environment variables missing in the jupyter notebook interface.

		These variables can be found in the `/etc/profile.d/shepherd-clients-env.sh` script

		In order to be able to connect to the Hadoop cluster you should modify the environment to include them.

		%% Cell type:markdown id:7c924d79-fea6-44cb-a1c4-e9b68ccada53 tags:

		## All-in-one setup

		Run the cell below to perform Kerberos authentication and environment modification at once

		%% Cell type:code id:0fb31d62-eb1f-4470-b4e2-76b9ea0669f1 tags:

		``` python
		import hadoop_utils
		```

		%% Output


		%% Cell type:markdown id:b0e5f37a-c071-4957-81c1-d82d53eb7dd1 tags:

		# Hadoop Command Line Interfaces (CLI)

		## Hadoop Distributed File System (HDFS) CLI

		HDFS can be managed using the `hdfs dfs` CLI

		%% Cell type:code id:85b56f27-1d30-4f14-ad37-8a1f34ef592a tags:

		``` python
		# Get the command help
		!hdfs dfs -help
		```

		%% Cell type:markdown id:e39ca9c4-5906-4149-9b7c-9a7bb613d4ef tags:

		### List content in HDFS

		%% Cell type:markdown id:56164ac4-c528-4764-a9d7-ae47d9bc9294 tags:

		`hdfs df -ls` to list the content & properties of a filesystem location

		%% Cell type:code id:823b0d6e-e496-4389-a950-81fec6f9fb47 tags:

		``` python
		# list the root
		!hdfs dfs -ls /
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		Found 6 items
		drwxr-xr-x - hdfs hadoop 0 2024-09-25 13:41 /apps
		drwxrwxrwx - hdfs hadoop 0 2024-10-01 19:12 /data
		drwxr-xr-x - hbase hadoop 0 2024-02-28 12:43 /hbase
		drwxrwxrwt - hdfs hadoop 0 2024-09-24 21:31 /tmp
		drwxr-xr-x - hdfs hadoop 0 2024-08-27 12:39 /user
		drwxrwxr-x - hive hadoop 0 2024-10-30 16:34 /warehouse

		%% Cell type:code id:f4e00d85-7e92-4f1e-899a-20c5d540eaf1 tags:

		``` python
		import getpass
		```

		%% Cell type:code id:173f3fda-f98c-491c-8ea5-7d18503db614 tags:

		``` python
		username = getpass.getuser()
		username
		```

		%% Output

		'torradeflot'

		%% Cell type:code id:b6dd2094-2308-44fe-bee8-43a3cb316bba tags:

		``` python
		# list the content of the user folder
		!hdfs dfs -ls /user/{username}
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		Found 5 items
		drwxr-xr-x - torradeflot hadoop 0 2023-12-19 07:42 /user/torradeflot/.Trash
		drwxr-xr-x - torradeflot hadoop 0 2023-12-19 07:43 /user/torradeflot/.sparkStaging
		drwxr-xr-x - torradeflot hadoop 0 2023-12-19 07:43 /user/torradeflot/data
		drwxr-xr-x - torradeflot hadoop 0 2024-01-29 11:11 /user/torradeflot/db
		-rw-r--r-- 1 torradeflot hadoop 1048576 2024-11-22 15:30 /user/torradeflot/test_1M

		%% Cell type:markdown id:e1e3b620-adb7-47cd-b886-d0f930c39d88 tags:

		### Upload / download to HDFS

		This can be done through the `hdfs dfs -put` and `hdfs dfs -get` commands

		%% Cell type:code id:003a3c0e-4921-4c34-9899-7f8da00ba6cd tags:

		``` python
		!dd if=/dev/urandom of=/tmp/test_1M bs=1024 count=1024
		```

		%% Output

		1024+0 records in
		1024+0 records out
		1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186929 s, 56.1 MB/s

		%% Cell type:code id:f3550a5e-affc-4a39-9b91-77e2d0a70177 tags:

		``` python
		!hdfs dfs -put /tmp/test_1M /user/{username}
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		2024-11-25 10:45:19,596 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable

		%% Cell type:code id:9605b336-8082-4c35-9866-3f2a6aaf27a0 tags:

		``` python
		!hdfs dfs -ls /user/{username}
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		Found 5 items
		drwxr-xr-x - torradeflot hadoop 0 2024-11-25 10:45 /user/torradeflot/.Trash
		drwxr-xr-x - torradeflot hadoop 0 2023-12-19 07:43 /user/torradeflot/.sparkStaging
		drwxr-xr-x - torradeflot hadoop 0 2023-12-19 07:43 /user/torradeflot/data
		drwxr-xr-x - torradeflot hadoop 0 2024-01-29 11:11 /user/torradeflot/db
		-rw-r--r-- 1 torradeflot hadoop 1048576 2024-11-25 10:45 /user/torradeflot/test_1M

		%% Cell type:code id:f45a5eb1-af49-4f2b-a637-f6df14bf73ea tags:

		``` python
		!hdfs dfs -get /user/{username}/test_1M /tmp/test_1M_copy
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		2024-11-25 10:39:25,751 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable

		%% Cell type:code id:6b8de8c6-2ef1-47f5-9863-ac5f3c674a18 tags:

		``` python
		!ls -la /tmp/test_1M_copy
		```

		%% Output

		-rw-r--r-- 1 torradeflot pic 1048576 Nov 25 10:39 /tmp/test_1M_copy

		%% Cell type:markdown id:5da7f601-e59d-4859-928a-30874033c340 tags:

		### Delete data from HDFS

		%% Cell type:code id:615a02a4-c5e1-4d1e-9128-d2c01a85c979 tags:

		``` python
		!hdfs dfs -rm /user/{username}/test_1M
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		2024-11-25 10:46:33,439 INFO fs.TrashPolicyDefault: Moved: 'hdfs://hsrv04.pic.es:9000/user/torradeflot/test_1M' to trash at: hdfs://hsrv04.pic.es:9000/user/torradeflot/.Trash/Current/user/torradeflot/test_1M1732527993433

		%% Cell type:markdown id:21ecd747-9b45-48a0-9fcd-dfe4e6e5557b tags:

		## Hive CLI (Beeline)

		Beeline is an interactive CLI to explore Hive. Since it is interactive it is a bit tricky to run it within python cells and get the stderr and stdout printed out.

		This is how it would look if you run it in a terminal

		### Open beeline

		[torradeflot@td835 hadoop-3.2.3]$ beeline
		...
		Connecting to jdbc:hive2://hsrv01.pic.es:10000/default;principal=hive/_HOST@PIC.ES
		Connected to: Apache Hive (version 3.1.2)
		Driver: Hive JDBC (version 3.1.2)
		Transaction isolation: TRANSACTION_REPEATABLE_READ
		Beeline version 3.1.2 by Apache Hive
		0: jdbc:hive2://hsrv01.pic.es:10000/default>

		Once the inside the beeline CLI we can explore the data in Hive

		### List databases

		0: jdbc:hive2://hsrv01.pic.es:10000/default> show databases;
		INFO : Compiling command(queryId=hive_20241125103556_8d796b62-687e-4069-86f5-98b2d77fbeb5): show databases
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241125103556_8d796b62-687e-4069-86f5-98b2d77fbeb5); Time taken: 0.074 seconds
		INFO : Executing command(queryId=hive_20241125103556_8d796b62-687e-4069-86f5-98b2d77fbeb5): show databases
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241125103556_8d796b62-687e-4069-86f5-98b2d77fbeb5); Time taken: 0.014 seconds
		INFO : OK
		+---------------------+
		\| database_name \|
		+---------------------+
		\| bruzzese \|
		\| castander \|
		\| cgonzale \|
		\| cosmohub \|
		\| default \|
		\| egonzale \|
		\| eriksen \|
		\| information_schema \|
		\| isevilla \|
		\| itutusau \|
		\| jcarrete \|
		\| jdelgado \|
		\| jflix \|
		\| mbreton \|
		\| msantama \|
		\| pgraciae \|
		\| rmetcalf \|
		\| rsanfeli \|
		\| sys \|
		\| tallada \|
		\| to_delete \|
		\| torradeflot \|
		\| udf \|
		\| vdeshpan \|
		\| wdassign \|
		+---------------------+
		25 rows selected (0.199 seconds)

		### Connect to a database

		e.g. cosmohub

		0: jdbc:hive2://hsrv01.pic.es:10000/default> use cosmohub;
		INFO : Compiling command(queryId=hive_20241125111214_f89a4615-20ef-4726-aa44-d5f5c1c6b91f): use cosmohub
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
		INFO : Completed compiling command(queryId=hive_20241125111214_f89a4615-20ef-4726-aa44-d5f5c1c6b91f); Time taken: 0.141 seconds
		INFO : Executing command(queryId=hive_20241125111214_f89a4615-20ef-4726-aa44-d5f5c1c6b91f): use cosmohub
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241125111214_f89a4615-20ef-4726-aa44-d5f5c1c6b91f); Time taken: 0.002 seconds
		INFO : OK
		No rows affected (0.149 seconds)

		### List tables

		0: jdbc:hive2://hsrv01.pic.es:10000/default> show tables;
		INFO : Compiling command(queryId=hive_20241125111313_9171df2e-60df-4e9f-9965-f8ba0d2dd269): show tables
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:tab_name, type:string, comment:from deserializer)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241125111313_9171df2e-60df-4e9f-9965-f8ba0d2dd269); Time taken: 0.013 seconds
		INFO : Executing command(queryId=hive_20241125111313_9171df2e-60df-4e9f-9965-f8ba0d2dd269): show tables
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241125111313_9171df2e-60df-4e9f-9965-f8ba0d2dd269); Time taken: 1.194 seconds
		INFO : OK
		+----------------------------------------------------+
		\| tab_name \|
		+----------------------------------------------------+
		\| abacussummit_base_c000_ph000_c \|
		\| alhambra \|
		\| alhambra_classified \|
		\| ben_results \|
		\| ben_results12 \|
		\| ben_results12_5 \|
		...

		### Describe a table

		Get column names and descriptions

		0: jdbc:hive2://hsrv01.pic.es:10000/default> describe des_dr1;
		INFO : Compiling command(queryId=hive_20241125111609_fde9bd8b-39df-43c4-ac4a-58e9a324f885): describe des_dr1
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:col_name, type:string, comment:from deserializer), FieldSchema(name:data_type, type:string, comment:from deserializer), FieldSchema(name:comment, type:string, comment:from deserializer)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241125111609_fde9bd8b-39df-43c4-ac4a-58e9a324f885); Time taken: 0.057 seconds
		INFO : Executing command(queryId=hive_20241125111609_fde9bd8b-39df-43c4-ac4a-58e9a324f885): describe des_dr1
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241125111609_fde9bd8b-39df-43c4-ac4a-58e9a324f885); Time taken: 0.013 seconds
		INFO : OK
		+-------------------------+------------+----------------------------------------------------+
		\| col_name \| data_type \| comment \|
		+-------------------------+------------+----------------------------------------------------+
		\| coadd_object_id \| bigint \| Unique identifier for the coadded objects \|
		\| tilename \| string \| Identifier of each one of the tiles on which the survey is gridded \|
		\| hpix_32 \| bigint \| Healpix identifier for its NSIDE grid size, in a NESTED schema \|
		\| hpix_64 \| bigint \| Healpix identifier for its NSIDE grid size, in a NESTED schema \|
		\| hpix_1024 \| bigint \| Healpix identifier for its NSIDE grid size, in a NESTED schema \|
		\| hpix_4096 \| bigint \| Healpix identifier for its NSIDE grid size, in a NESTED schema \|
		\| hpix_16384 \| bigint \| Healpix identifier for its NSIDE grid size, in a NESTED schema \|
		...

		### Perform a query

		0: jdbc:hive2://hsrv01.pic.es:10000/default> select count(*) from des_dr1;
		INFO : Compiling command(queryId=hive_20241125111917_ae2b91e4-5969-4cea-b1f7-a2340f424932): select count(*) from des_dr1
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241125111917_ae2b91e4-5969-4cea-b1f7-a2340f424932); Time taken: 3.524 seconds
		INFO : Executing command(queryId=hive_20241125111917_ae2b91e4-5969-4cea-b1f7-a2340f424932): select count(*) from des_dr1
		INFO : Completed executing command(queryId=hive_20241125111917_ae2b91e4-5969-4cea-b1f7-a2340f424932); Time taken: 0.001 seconds
		INFO : OK
		+------------+
		\| _c0 \|
		+------------+
		\| 399263026 \|
		+------------+
		1 row selected (3.561 seconds)

		### Exit

		0: jdbc:hive2://hsrv01.pic.es:10000/default> !quit
		Closing: 0: jdbc:hive2://hsrv01.pic.es:10000/default;principal=hive/_HOST@PIC.ES


		%% Cell type:markdown id:ab4f87db-bf80-4fbd-ba7a-3bb41e560384 tags:

		## Running Hive CLI from a notebook

		<p style="font-size:36px; color:red; font-weight:bold">Hack alert</p>

		This has been done for demostrative purposes but it is not convenient. Use beeline in a shell or a proper python client to submit queries to Hive

		Since the Hive CLI is interactive, it is not possible to run it from jupyter notebook. However, we can run it in a python subprocess and pipe the stdin, stdout, stderr.

		The next cell is an equivalent to launching a Beeline CLI in the shell

		%% Cell type:code id:fdd29869-75e3-43da-9cf5-5b65e2ea6386 tags:

		``` python
		beeline = hadoop_utils.BeelineCLI()
		```

		%% Output

		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.

		%% Cell type:code id:3cdd821c-c3ab-4327-89df-3d50ccadaaf2 tags:

		``` python
		beeline.run('show databases;')
		```

		%% Output

		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		SLF4J: Class path contains multiple SLF4J bindings.
		SLF4J: Found binding in [jar:file:/opt/tez-0.10.1/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hive-3.1.2/lib/log4j-slf4j-impl-2.10.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: Found binding in [jar:file:/opt/hadoop-3.2.3/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
		SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
		SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
		Connecting to jdbc:hive2://hsrv01.pic.es:10000/default;principal=hive/_HOST@PIC.ES
		Connected to: Apache Hive (version 3.1.2)
		Driver: Hive JDBC (version 3.1.2)
		Transaction isolation: TRANSACTION_REPEATABLE_READ
		Beeline version 3.1.2 by Apache Hive
		0: jdbc:hive2://hsrv01.pic.es:10000/default> show databases;
		INFO : Compiling command(queryId=hive_20241126092216_1692f891-5ea7-4255-a87b-9e5433fdb438): show databases
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:database_name, type:string, comment:from deserializer)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241126092216_1692f891-5ea7-4255-a87b-9e5433fdb438); Time taken: 0.188 seconds
		INFO : Executing command(queryId=hive_20241126092216_1692f891-5ea7-4255-a87b-9e5433fdb438): show databases
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241126092216_1692f891-5ea7-4255-a87b-9e5433fdb438); Time taken: 0.015 seconds
		INFO : OK
		+---------------------+
		\| database_name \|
		+---------------------+
		\| bruzzese \|
		\| castander \|
		\| cgonzale \|
		\| cosmohub \|
		\| default \|
		\| egonzale \|
		\| eriksen \|
		\| information_schema \|
		\| isevilla \|
		\| itutusau \|
		\| jcarrete \|
		\| jdelgado \|
		\| jflix \|
		\| mbreton \|
		\| msantama \|
		\| pgraciae \|
		\| rmetcalf \|
		\| rsanfeli \|
		\| sys \|
		\| tallada \|
		\| to_delete \|
		\| torradeflot \|
		\| udf \|
		\| vdeshpan \|
		\| wdassign \|
		+---------------------+
		25 rows selected (0.415 seconds)

		%% Cell type:code id:9f183e79-8f7e-4741-99ce-9be81834144e tags:

		``` python
		beeline.run("use cosmohub;")
		```

		%% Output

		0: jdbc:hive2://hsrv01.pic.es:10000/default> use cosmohub;
		INFO : Compiling command(queryId=hive_20241125112922_eac834d3-310c-4ad1-a5d7-8ef0fa7814f3): use cosmohub
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
		INFO : Completed compiling command(queryId=hive_20241125112922_eac834d3-310c-4ad1-a5d7-8ef0fa7814f3); Time taken: 0.012 seconds
		INFO : Executing command(queryId=hive_20241125112922_eac834d3-310c-4ad1-a5d7-8ef0fa7814f3): use cosmohub
		INFO : Starting task [Stage-0:DDL] in serial mode
		INFO : Completed executing command(queryId=hive_20241125112922_eac834d3-310c-4ad1-a5d7-8ef0fa7814f3); Time taken: 0.002 seconds
		INFO : OK
		No rows affected (0.02 seconds)

		%% Cell type:code id:c3d5fa90-170f-4cdc-a3c7-71e696ea968d tags:

		``` python
		beeline.run("select count(*) from des_dr1;", 5)
		```

		%% Output

		0: jdbc:hive2://hsrv01.pic.es:10000/default> select count(*) from des_dr1;
		INFO : Compiling command(queryId=hive_20241125113125_aba6ec07-bff3-43d0-a174-835cd29e7f7e): select count(*) from des_dr1
		INFO : Semantic Analysis Completed (retrial = false)
		INFO : Returning Hive schema: Schema(fieldSchemas:[FieldSchema(name:_c0, type:bigint, comment:null)], properties:null)
		INFO : Completed compiling command(queryId=hive_20241125113125_aba6ec07-bff3-43d0-a174-835cd29e7f7e); Time taken: 3.418 seconds
		INFO : Executing command(queryId=hive_20241125113125_aba6ec07-bff3-43d0-a174-835cd29e7f7e): select count(*) from des_dr1
		INFO : Completed executing command(queryId=hive_20241125113125_aba6ec07-bff3-43d0-a174-835cd29e7f7e); Time taken: 0.001 seconds
		INFO : OK
		1 row selected (3.456 seconds)
		+------------+
		\| _c0 \|
		+------------+
		\| 399263026 \|
		+------------+

		%% Cell type:code id:f6b73783-d08e-4985-862a-3caeac53adc6 tags:

		``` python
		```