SQL good video

Advertisements

hadoop:sqoop import into hive table

This will load data into existing table temp10

sqoop import –connect jdbc:mysql://localhost/mysql –table temp  –hive-import –hive-table temp10 –username root -m 1;

hive> select * from temp10;
OK
10    10
10    20
Time taken: 0.242 seconds, Fetched: 2 row(s)

this will create new hive table tempnew and imports data into it

sqoop import –connect jdbc:mysql://localhost/mysql –table temp  –hive-import –hive-table tempnew –create-hive-table –username root -m 1;

hive> select * from tempnew;
OK
10    10
10    20
Time taken: 0.324 seconds, Fetched: 2 row(s)
hive>

http://sqoop.apache.org/docs/1.4.5/SqoopUserGuide.html#_importing_data_into_hive

Argument Description
--hive-home <dir> Override $HIVE_HOME
--hive-import Import tables into Hive (Uses Hive’s default delimiters if none are set.)
--hive-overwrite Overwrite existing data in the Hive table.
--create-hive-table If set, then the job will fail if the target hive
table exits. By default this property is false.
--hive-table <table-name> Sets the table name to use when importing to Hive.
--hive-drop-import-delims Drops \n, \r, and \01 from string fields when importing to Hive.
--hive-delims-replacement Replace \n, \r, and \01 from string fields with user defined string when importing to Hive.
--hive-partition-key Name of a hive field to partition are sharded on
--hive-partition-value <v> String-value that serves as partition key for this imported into hive in this job.
--map-column-hive <map> Override default mapping from SQL type to Hive type for configured columns.

 

RHEL 7 – Notes: NTP:Chronyd

Update ntp server in RHEL7 using new chronyd.service

vi /etc/chrony.conf
systemctl restart chronyd.service
systemctl enable chronyd.service
chronyc sources -v
#timedatectl
Local time: Tue 2015-12-22 04:21:22 EST
Universal time: Tue 2015-12-22 09:21:22 UTC
RTC time: Tue 2015-12-22 09:21:22
Time zone: America/New_York (EST, -0500)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: no
Last DST change: DST ended at
Sun 2015-11-01 01:59:59 EDT
Sun 2015-11-01 01:00:00 EST
Next DST change: DST begins (the clock jumps one hour forward) at
Sun 2016-03-13 01:59:59 EST
Sun 2016-03-13 03:00:00 EDT

#timedatectl list-timezones
Africa/Abidjan
Africa/Accra
Africa/Addis_Ababa
Africa/Algiers
Africa/Asmara
Africa/Bamako

#How to change time zone

#timedatectl
Local time: Tue 2015-12-22 04:23:16 EST
Universal time: Tue 2015-12-22 09:23:16 UTC
RTC time: Tue 2015-12-22 09:23:15
       Time zone: America/New_York (EST, -0500)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: no
Last DST change: DST ended at
Sun 2015-11-01 01:59:59 EDT
Sun 2015-11-01 01:00:00 EST
Next DST change: DST begins (the clock jumps one hour forward) at
Sun 2016-03-13 01:59:59 EST
Sun 2016-03-13 03:00:00 EDT

# timedatectl set-timezone Asia/Singapore

# timedatectl
Local time: Tue 2015-12-22 17:23:29 SGT
Universal time: Tue 2015-12-22 09:23:29 UTC
RTC time: Tue 2015-12-22 09:23:29
Time zone: Asia/Singapore (SGT, +0800)
NTP enabled: yes
NTP synchronized: yes
RTC in local TZ: no
DST active: n/a

RHEL 7 – Notes: Journalctl

Journalctl to view system logs:

===========================

journalctl -n
journalctl -n 5
journalctl -p err
journalctl –since today
journalctl –since “2015-12-22 02:00:00” –until “2015-12-22 03:00:00”
journalctl -o verbose

journalctl _PID=1

journalctl -b (since last reboot if they were made persistent)

 

How to make persistent:

mkdir /var/log/journal

chown root:systemd-journal /var/log/journal

chmod 2755 /var/log/journal

kilall -USR1 systemd-journald

 

How to drop cache in Linux

echo 3 > /proc/sys/vm/drop_caches

https://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txthttps://www.kernel.org/doc/Documentation/sysctl/vm.txt
drop_caches

Writing to this will cause the kernel to drop clean caches, as well as
reclaimable slab objects like dentries and inodes.  Once dropped, their
memory becomes free.

To free pagecache:
	echo 1 > /proc/sys/vm/drop_caches
To free reclaimable slab objects (includes dentries and inodes):
	echo 2 > /proc/sys/vm/drop_caches
To free slab objects and pagecache:
	echo 3 > /proc/sys/vm/drop_caches