Monday, July 17, 2017

HIVE load csv text file

First column must not contain spaces, otherwise it will load NULL! Intermediate columns can contain spaces, it is suggested to load them as String and then convert to the required type.
1.Load into special text table
CREATE TABLE tt (
  id BIGINT,
  data STRING
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t'
STORED AS TEXTFILE
2.Move to the normal table

Tuesday, May 9, 2017

Hive QL vs SQL


No "IN" Clause
Zero-based indexing
"AS" aliasing for columns only


SQL





HIVE QL
ORDER BY
SELECT * FROM t WHERE id in (...)
SORT BY (ORDER BY is partial sort)
SELECT * FROM t LEFT SEMI JOIN t2 ON ...
SELECT f[1] FROM t SELECT f[0] FROM t
SELECT * FROM t AS tt                     SELECT * FROM t tt


Tuesday, May 2, 2017

Mercurial commands

Create a branch
hg bookmark [name]
Reset local changes
hg revert --all
Fetch latest code
hg pull
Fetch latest code and update local code
hg update master
Commit
hg commit [-m "message"]
Update commit with new changes
hg amend [!! use of -m "message" overwrites the log]
Status
hg diff
hg status
Current bookmark (branch)
hg identify
Switch bookmark and rebase
hg update [name]
Difference from the rev
hg diff -r [rev_num]
Use file from a revision
hg revert -r [rev_num] [file]
Use theirs or yours version on rebase
hg resolve -t internal:other --all
hg resolve -t internal:local --all
Use manual resolve on rebase

hg resolve -m [file]