36 posts

AVRO Essentials

From Avro Apache Documentation source: Apache Avro™ is a data serialization system. Avro provides: Schemas Avro relies on schemas. When Avro data is read, the schema used when writing it is always present. This permits each datum to be written with no per-value overheads, making serialization both fast and […]

JSON Schema essentials

Quick Q-A (source – Goog directly) What is a JSON Schema used for? Primarily, JSON Schema is used to validate data. The sorts of data JSON Schema can validate is data encoded in JSON. JSON stands for JavaScript Object Notation and is a subset of Javascript. JSON is both human […]

Handling JSON with SQL Server

— using — ISJSON(): we can check valid JSON using this function SELECT TOP (1000) [gscData],[gscSiteUrl],[gscDate],[insDate],[insDateUTC],ISJSON(convert(nvarchar, gscData)) as is_jsonFROM [dwhlite].[gsc].[dat_google_search_console]; — tk: json functions in SQL Server— ISJSON(): we can check valid JSON using this function— JSON_VALUE(): It extracts a scalar value from the JSON data— JSON_MODIFY(): It modifies […]

Python tutorial – w3schools based

This is my learning log from w3schools python tutorial Python HOME (todo) todo Python Intro Python Get Started Python Syntax Python Comments Python Variables Python Variables Variable names Asigning multiple variables Output Variables Global variables Python Data Types Python Numbers […]

numpy and pandas install problems

numpy and pandas install problems Failed building wheel for numpyRunning clean for numpyComplete output from command “C:\Users.……..\PRO\stocks\venv\Scripts\python.exe” -u -c “import setuptools, tokenize;_ file_=’C:\Users.….\AppData\Local\Temp\pip-install-pxslxiaw\numpy\’;f=getattr(tokenize, ‘open’, open)(file);‘\r\n’, ‘\n’);f.close();exec(compile(code, file, ‘exec’))” clean –all:Running from numpy source directory. clean is not supported, use one of the following instead: Add –force to your […]

HIVE – return closest Monday before date

How to return closest Monday date before, when date is given? It is very simple, user next_day and date_add funtions select next_day(date_add(‘YOUR_DATE’,-7),’MONDAY’) Check — first next Monday od selected date select next_day(‘2020-10-21’, ‘MONDAY’); — returns closest Monday date before select next_day(date_add(‘2020-11-15′,-7),’MONDAY’) select next_day(date_add(‘2020-11-16′,-7),’MONDAY’) select next_day(date_add(‘2020-11-17′,-7),’MONDAY’) select next_day(date_add(‘2020-11-18′,-7),’MONDAY’) select next_day(date_add(‘2020-11-19′,-7),’MONDAY’) select […]

How to install python on SSH

Install Python on web server via SSH Get latest version of Python. Here you can see versions: wget Decompress files: tar xvzf Python-3.9.0.tgz Open decompressed directory: cd Python-3.9.0 Install: ./configure –prefix=$HOME/.local Make: make Make install – will install pip and setup tools: make install Go to bash profile […]

Python googletrans

How to user googletrans # pip install googletrans # # If we do not specify the source and the destination languages, # googletrans tries to detect the language and translates it into English. import googletrans from googletrans import Translator print(googletrans.LANGUAGES) # without language source and destination choose #translator = […]

List of Fibonacci in Python and Fibonacci function

Fibonnaci in python by token: # fibbonaci: fib_list = [0, 1] for i in range(10): fib_list.append(fib_list[-2] + fib_list[-1]) print(fib_list) Or if You want Python function to generate Fibonacci def funFibonaci(numberElements_in): fib_list = [0, 1] if numberElements_in == 1: return fib_list[-2] if numberElements_in == 2: return fib_list[-1] for i in range(numberElements_in-2): […]

Python list of dates between two dates

s_date = datetime.strptime(“2020-01-01”, “%Y-%m-%d”)e_date = datetime.strptime(“2020-01-15”, “%Y-%m-%d”)delta = e_date – s_datefor i in range(delta.days + 1): day = s_date + timedelta(days=i) l.insert(i, day) #print(day)print(l) or you can define function from datetime import datetime, timedeltas_date = datetime.strptime(“2020-01-01”, “%Y-%m-%d”)e_date = datetime.strptime(“2020-01-15”, “%Y-%m-%d”)def out_date_range(argStartDate, argEndDate): delta = argEndDate – argStartDate l = [] […]

Request library in Python

–– coding: utf-8 –– “”” Created on Fri Dec 13 22:26:18 2019 @author: token “”” libs import requests strUrlAddress = ‘’ get all url html data as text requests.get(strUrlAddress).text test a = requests.get(strUrlAddress) return html data a.text return encoding a.encoding json with server data , text lenthetc., meta, length […]