@h12.io

ua: User-Agent parser in Go

18 July 2016

Design

The UA detection algorithm is a pipeline containing 3 steps:

  1. Scan: scan products and its corresponding comments from a UA string.
  2. Parse: parse device info from products.
  3. Detect: detect device model by looking up in a database and get related info, e.g. screen resolution.

Scan

The scanner scans the user agent string into products. Each product has its own name, version and comments. e.g.

For user agent string:

Mozilla/5.0 (Linux; U; Android ROM v3; en-us; ALCATEL ONE TOUCH 991 Build/GRK39F) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

Its products are:

  • Mozilla/5.0
    • Linux
    • U
    • Android ROM v3
    • en-us
    • ALCATEL ONE TOUCH 991 Build/GRK39F
  • AppleWebKit/533.1
    • KHTML, like Gecko
  • Version/4.0
  • Mobile
  • Safari/533.1

Parse

The parser then parses the scanned products and fill the information into a Device struct, e.g.

Mozilla:  5.0
Linux:    true
Security: U
Android:  ROM v3
Locale:   en-us
Model:    ALCATEL ONE TOUCH 991
Build:    GRK39F
Webkit:   533.1
Version:  4.0
Mobile:   true
Safari:   533.1

Detect

The detector is then able to use the parsed Device structure to detect the device type & related info: vendor, screen resolution, tablet-or-not, etc, by searching in a device database.

Design Goals

Accuracy

The accuracy of the detection is determined by the completeness of the device database, then we might have to migrate the data from multiple sources. Automatic tests will be adapted from those resources.

References

Detectors

Parsers

A Manager's FAQ

3 July 2016

This is an insightful list from Henry Ward:

  • How do I get employees to perform better? Tell them what they are doing well.
  • How do I give negative feedback? By being curious.
  • How do I decide what to delegate? Delegate the work you want to do.
  • How should I prioritize? Fix problems. Then prevent problems.
  • How should I grade employees? Don’t. Teach them to self-evaluate.
  • When do I fire somebody? When you know they can’t succeed.
  • How do I fire somebody? By apologizing for our failures.
  • Why can’t I just tell people what to do? Because the more responsibility you have, the less authority you have.
  • How do I know if I am a good manager? Employees ask you for advice.
  • How do I know if I have good management team? Shit rolls uphill.

On Reliable Persistence

2 July 2016

A short investigation on reliable persistence:

  • Crash consistency is hard but possible at a huge performance cost

  • Data safty must rely on distributed solution

  • Perhaps we should just give up manual fsync and rely on

    • OS background flush
    • corruption detection & correction at startup
    • replication

Note on OpenWRT

27 February 2016

Obtain OpenWRT

  • Find the device model at the Table of Hardware (Huawei HG556a C)
  • Open Device Techdata for Firmware OpenWrt Install URL
  • Open Device Page for Installation instructions

Computer - Cable - Router

Connect the computer to the router with a cable.

Install OpenWRT

Follow the instructions to install OpenWRT.

SSH

First login:

telnet 192.168.1.1

Change password to enable SSH:

passwd
exit

For public key authentication, add the public key:

ssh-copy-id [email protected]
ssh [email protected]
mv ~/.ssh/authorized_keys /etc/dropbear
exit

WiFi

In /etc/config/wireless:

config wifi-device  radio0
	......
	# REMOVE THIS LINE TO ENABLE WIFI:
	# option disabled 1

config wifi-iface
	option device     radio0
	option network    lan
	option mode       ap
	option encryption psk2
	option hidden     1
	option ssid       [SSID]
	option key        [WiFi password]
	option macaddr    [00:0A:4B:3C:6D:02]

Execute:

wifi

Computer - WiFi - Router

Disconnect the cable to the router. Try SSH via WiFi.

Internet

In /etc/config/network:

It is better to change the lan address so that it will not be conflicted with the upstream address.

config interface 'lan'
	option ipaddr '192.168.2.1'
# lan switch
config switch_vlan
	option device 	eth0
	option vlan 	1
	option ports 	"1 2 5t"

# wan switch
config switch_vlan
	option device 	eth0
	option vlan 	2
	option ports 	"0 5t"

For DHCP wan:

config interface wan
	option ifname eth0.2
	option proto  dhcp

For PPPOE wan:

config interface wan
	option ifname   eth0.2
	option proto    pppoe
	option username [PPPOE user name]
	option password [password]

Computer - WiFi - Router - Cable - Internet

Connect the internet cable to the router, reboot the router.

About Pretty Printing

1 February 2016

JSON

cat xxx.json | jq .

XML

cat xxx.xml | xmllint --format -

Sending Email from Gmail using Go

31 January 2016
  1. Turn on “2-Step Verification” so that an “App password” can be generated

  2. go get gopkg.in/gomail.v2

  3. Send your Email like the example below:

package main

import (
	"gopkg.in/gomail.v2"
)

func main() {
	m := gomail.NewMessage()
	m.SetHeader("From", "[email protected]")
	m.SetAddressHeader("To", "[email protected]", "to_name")
	m.SetAddressHeader("Cc", "[email protected]", "cc_name")
	m.SetHeader("Subject", "Hello! TEST!")
	m.SetBody("text/html", "Hello! <b>TEST</b>!")

	d := gomail.NewPlainDialer("smtp.gmail.com", 587, "[email protected]", "the App password")

	if err := d.DialAndSend(m); err != nil {
		panic(err)
	}
}

SSH Resources in Go

31 January 2016
  • golang.org/x/crypto/ssh
  • github.com/YuriyNasretdinov/GoSSHa

Articles: