From Avro Apache Documentation source: https://avro.apache.org/docs/ Apache Avro™ is a data serialization system. Avro provides: Schemas Avro relies on schemas. When Avro data is read, the schema used when writing it is always present. This permits each datum to be written with no per-value overheads, making serialization both fast and […]
token
36 posts
Quick Q-A (source – Goog directly) What is a JSON Schema used for? Primarily, JSON Schema is used to validate data. The sorts of data JSON Schema can validate is data encoded in JSON. JSON stands for JavaScript Object Notation and is a subset of Javascript. JSON is both human […]
Good course: Part1: https://www.youtube.com/watch?v=QpdhBUYk7Kk think always minimum and maximum Part2: https://www.youtube.com/watch?v=-CuY5ADwn24 PK & FK on same column Bridge table: Datatypes
— using https://www.sqlshack.com/the-json_query-function-to-extract-objects-from-json-data/ — ISJSON(): we can check valid JSON using this function SELECT TOP (1000) [gscData],[gscSiteUrl],[gscDate],[insDate],[insDateUTC],ISJSON(convert(nvarchar, gscData)) as is_jsonFROM [dwhlite].[gsc].[dat_google_search_console]; — tk: json functions in SQL Server— ISJSON(): we can check valid JSON using this function— JSON_VALUE(): It extracts a scalar value from the JSON data— JSON_MODIFY(): It modifies […]
https://stackoverflow.com/questions/5210309/how-can-i-enable-the-federated-engine-in-mysql-after-installation
This is my learning log from w3schools python tutorial Python HOME (todo) todo Python Intro https://www.w3schools.com/python/python_intro.asp Python Get Started https://www.w3schools.com/python/python_getstarted.asp Python Syntax https://www.w3schools.com/python/python_syntax.asp Python Comments https://www.w3schools.com/python/python_comments.asp Python Variables Python Variables https://www.w3schools.com/python/python_variables.asp Variable names https://www.w3schools.com/python/python_variables_names.asp Asigning multiple variables https://www.w3schools.com/python/python_variables_multiple.asp Output Variables https://www.w3schools.com/python/python_variables_output.asp Global variables https://www.w3schools.com/python/python_variables_global.asp Python Data Types https://www.w3schools.com/python/python_datatypes.asp Python Numbers […]
using mysql-connector-python package
Using mysql-connector-python
https://realpython.com/python-json/
Python JSON dict object list, tuple array str string int, long, float number True true False false None null
https://stackoverflow.com/questions/11319909/creating-new-variables-in-loop-with-names-from-list-in-python I think dictionaries are more suitable for this purpose: fly you can use globals()[]:
1) Windows Start Button > Run2) Type cmd and hit enter (“command” in Win 10)3) Go to the folder with the CSV files 4) Type copy *.csv merge.txt and hit enter to copy all data in the files into merge.txt.
numpy and pandas install problems Failed building wheel for numpyRunning setup.py clean for numpyComplete output from command “C:\Users.……..\PRO\stocks\venv\Scripts\python.exe” -u -c “import setuptools, tokenize;_ file_=’C:\Users.….\AppData\Local\Temp\pip-install-pxslxiaw\numpy\setup.py’;f=getattr(tokenize, ‘open’, open)(file);code=f.read().replace(‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” clean –all:Running from numpy source directory. setup.py clean is not supported, use one of the following instead: Add –force to your […]
Uprgade pip
Local copy existing extractUse local copy source Close old extract (optional)Edit data source based on local copy
How to set other data location on s3? Just use ALTER TABLE. The example code is losted here: alter table d_ppo_category_pc_mapping set location ‘s3://YOUR_NEW_S3_LOCATION’;
How to get first day of quarter in hive? select add_months(trunc(‘2020-10-01′,’MM’),-(month(‘2020-10-01’)-1)%3)
How to return closest Monday date before, when date is given? It is very simple, user next_day and date_add funtions select next_day(date_add(‘YOUR_DATE’,-7),’MONDAY’) Check — first next Monday od selected date select next_day(‘2020-10-21’, ‘MONDAY’); — returns closest Monday date before select next_day(date_add(‘2020-11-15′,-7),’MONDAY’) select next_day(date_add(‘2020-11-16′,-7),’MONDAY’) select next_day(date_add(‘2020-11-17′,-7),’MONDAY’) select next_day(date_add(‘2020-11-18′,-7),’MONDAY’) select next_day(date_add(‘2020-11-19′,-7),’MONDAY’) select […]
Install Python on web server via SSH Get latest version of Python. Here you can see versions: https://www.python.org/ftp/python/ wget https://www.python.org/ftp/python/3.9.0/Python-3.9.0.tgz Decompress files: tar xvzf Python-3.9.0.tgz Open decompressed directory: cd Python-3.9.0 Install: ./configure –prefix=$HOME/.local Make: make Make install – will install pip and setup tools: make install Go to bash profile […]
How to user googletrans # pip install googletrans # http://zetcode.com/python/googletrans/ # If we do not specify the source and the destination languages, # googletrans tries to detect the language and translates it into English. import googletrans from googletrans import Translator print(googletrans.LANGUAGES) # without language source and destination choose #translator = […]
Fibonnaci in python by token: # fibbonaci: fib_list = [0, 1] for i in range(10): fib_list.append(fib_list[-2] + fib_list[-1]) print(fib_list) Or if You want Python function to generate Fibonacci def funFibonaci(numberElements_in): fib_list = [0, 1] if numberElements_in == 1: return fib_list[-2] if numberElements_in == 2: return fib_list[-1] for i in range(numberElements_in-2): […]
select slot, collect_set(size) from (select slot, size, count(d_date_id) as cnt from xxx.table1 where dt >= 20200601 group by slot, size ) x group by slot ;
here is an answer https://answers.microsoft.com/en-us/msoffice/forum/all/excel-or-word-desktop-keeps-asking-me-to-login/cf0308f7-8d98-4139-92b5-deb8139a1bb8
If you want to stop script for a while use code listed below. time.sleep(1) – sleep for a one second import timefor i in range(100): time.sleep(1) print(i)
s_date = datetime.strptime(“2020-01-01”, “%Y-%m-%d”)e_date = datetime.strptime(“2020-01-15”, “%Y-%m-%d”)delta = e_date – s_datefor i in range(delta.days + 1): day = s_date + timedelta(days=i) l.insert(i, day) #print(day)print(l) or you can define function from datetime import datetime, timedeltas_date = datetime.strptime(“2020-01-01”, “%Y-%m-%d”)e_date = datetime.strptime(“2020-01-15”, “%Y-%m-%d”)def out_date_range(argStartDate, argEndDate): delta = argEndDate – argStartDate l = [] […]
# first on numbers in range rng = range(15)print(rng)l = [*rng]print(l)
alter table table_name change column column_name column_name column_name_type after column_name2; https://community.cloudera.com/t5/Support-Questions/How-can-we-change-the-column-order-in-Hive-table-without/td-p/89495
What is SQL Server? SQL Server is a relational database management system (RDBMS) from Microsoft. In a very simple sense, it is a database system, by means of which we manage database elements such as tables, views, programming elements. SQL Server is currently the flagship database server offered by Microsoft. […]
To change table name in HIVE use code listed below: ALTER TABLE table_name RENAME TO new_table_name;
select str_to_map(column,’,’,’:’) from table ;
–– coding: utf-8 –– “”” Created on Fri Dec 13 22:26:18 2019 https://2.python-requests.org//pl/latest/user/quickstart.html @author: token “”” libs import requests strUrlAddress = ‘https://amazon.com’ get all url html data as text requests.get(strUrlAddress).text test a = requests.get(strUrlAddress) return html data a.text return encoding a.encoding json with server data , text lenthetc., meta, length […]
https://www.wikihow.com/Use-Windows-Command-Prompt-to-Run-a-Python-File
Python short tutorial will improve your programming skills very quickly.