balkian/balkian.github.com
1
0
Fork 0
mirror of https://github.com/balkian/balkian.github.com.git synced 2025-08-23 20:22:20 +00:00
Code Issues Releases Wiki Activity

Move to Hugo

Browse Source
This commit is contained in:
J. Fernando Sánchez
2019-01-10 20:26:53 +01:00
parent dbe35d2dde
commit 7c3fb50312
219 changed files with 19924 additions and 18981 deletions
Download Patch File Download Diff File Expand all files Collapse all files

53
content/post/2013-08-17-creating-my-web.md Normal file
View File

@@ -0,0 +1,53 @@
---
title: Creating my web
date: 2013-08-22 14:14:22
tags:
- starters
- javascript
- ruby
- github
- git
---
I've finally decided to set up a decent personal page. I have settled
for github-pages because I like the idea of keeping my site in a
repository and having someone else host and deploy it for me. The site
will be really simple, mostly static files. Thanks to Github,
[Jekyll](http://jekyllrb.com) will automatically generate static pages
for my posts every time I commit anything new to this repository.
But Jekyll can be used independently, so if I ever choose to host the
site myself, I can do it quite easily. Another thing that I liked about
this approach is that the generated html files can be used in the
future, and I will not need Jekyll to serve it. Jekyll is really simple
and most of the things are written in plain html. That means that
everything could be easily reused if I ever choose to change to another
blogging framework (e.g. pelical). But, for the time being, I like the
fact that Github takes care of the compilation as well, so I can simply
modify or add files through the web interface should I need to.
I hadn't played with HTML and CSS for a while now, so I also wanted to
use this site as a playground. At some point, I realised I was doing
mostly everything in plain HTML and CSS, and decided to keep it like
that for as long as possible. As of this writing, I haven't included
any Javascript code in the page. Probably I will use some to add my
[gists](http://gist.github.com/balkian) and
[repositories](http://github.com/balkian), but we will see about that.
I think the code speaks for itself, so you can check out [my repository
on Github](http://github.com/balkian/balkian.github.com). You can clone
and deploy it easily like this:
```bash
git clone
https://github.com/balkian/balkian.github.com cd balkian.github.com
jekyll serve -w
```
I will keep updating this post with information about:
- Some Jekyll plugins that might be useful
- What CSS tricks I learnt
- The webfonts I used
- The badge on the left side of the page

13
content/post/2013-08-22-Remove-git-files-with-globbing.md Normal file
View File

@@ -0,0 +1,13 @@
---
title: Remove git files with globbing
date: 2013-08-22 23:14:00
tags:
- git
---
A simple trick. If you want to remove all the '.swp' files from a git
repository, just use:
```bash
git rm --cached '**.swp'
```

120
content/post/2014-03-27-updating-eurolovemap.md Normal file
View File

@@ -0,0 +1,120 @@
---
title: Updating EuroLoveMap
date: 2014-03-27 14:00:00
tags:
- javascript
- python
- heroku
---
As part of the [OpeNER
hackathon](http://www.opener-project.org/2013/07/18/opener-hackathon-in-amsterdam/)
we decided to build a prototype that would allow us to compare how
different countries feel about several topics. We used the OpeNER
pipeline to get the sentiment from a set of newspaper articles we
gathered from media in several languages. Then we aggregated those
articles by category and country (using the source of the article or the
language it was written in), obtaining the "overall feeling" of each
country about each topic. Then, we used some fancy JavaScript to make
sense out of the raw information.
It didn't go too bad, it turns out [we
won](http://eurosentiment.eu/wp-content/uploads/2013/07/BOLv9qnCIAAJEek.jpg).
Now, it was time for a face-lift. I used this opportunity to play with
new technologies and improve it:
- Using Flask, this time using python 3.3 and Bootstrap 3.0
- Cool HTML5+JS cards (thanks to
[pastetophone](http://pastetophone.com))
- Automatic generation of fake personal data to test the interface
- Obfuscation of personal emails
The result can be [seen here](http://eurolovemap.herokuapp.com/).
Publishing a Python 3 app on Heroku
-----------------------------------
```bash
mkvirtualenv -p /usr/bin/python3.3 eurolovemap
```
Since Heroku uses python 2.7 by default, we have to tell it which
version we want, although it supports python 3.4 as well. I couldn't
get python 3.4 working using the
[deadsnakes](https://launchpad.net/~fkrull/+archive/deadsnakes) ppa, so
I used python 3.3 instead, which works fine but is not officially
supported. Just create a file named *runtime.txt* in your project root,
with the python version you want to use:
```bash
python-3.3.1
```
Don't forget to freeze your dependencies so Heroku can install them:
`bash pip freze > requirements.txt`
Publishing personal emails
--------------------------
There are really sophisticated and effective ways to obfuscate personal
emails so that spammers cannot easily grab yours. However, this time I
needed something really simple to hide our emails from the simplest form
of crawlers. Most of the team are in academia somehow, so in the end all
our emails are available in sites like Google Scholar. Anyway, nobody
likes getting spammed so I settled for a custom [Caesar
cipher](http://en.wikipedia.org/wiki/Caesar_cipher). Please, don't use
it for any serious application if you are concerned about being spammed.
```python
def blur_email(email):
return "".join([chr(ord(i)+5) for i in email])
```
And this is the client side:
```javascript
window.onload = function(){
elems = document.getElementsByClassName('profile-email');
for(var e in elems){
var blur = elems[e].innerHTML;
var email = "";
for(var s in blur){
var a = blur.charCodeAt(s)
email = email+String.fromCharCode(a-5);
}
elems[e].innerHTML = email;
}
}
```
Unfortunately, this approach does not hide your email from anyone using
[PhantomJS](http://phantomjs.org/),
[ZombieJS](http://zombie.labnotes.org/) or similar. For that, other
approaches like generating a picture with the address would be
necessary. Nevertheless, it is overkill for a really simple ad-hoc
application with custom formatting and just a bunch of emails that would
easily be grabbed manually.
Generation of fake data
-----------------------
To test the contact section of the site, I wanted to populate it with
fake data. [Fake-Factory](https://github.com/joke2k/faker) is an amazing
library that can generate fake data of almost any kind: emails,
association names, acronyms... It even lets you localise the results
(get Spanish names, for instance) and generate factories for certain
classes (à la Django).
But I also wanted pictures, enter [Lorem Pixel](http://lorempixel.com/).
With its API you can generate pictures of almost any size, for different
topics (e.g. nightlife, people) and with a custom text. You can even use
an index, so it will always show the same picture.
For instance, the picture below is served through Lorem Pixel.
![](http://lorempixel.com/400/200/nightlife/)
By the way, if you only want cat pictures, take a look at
[Placekitten](http://placekitten.com/). And for NSFW text, there's the
[Samuel L. Jackson Ipsum](http://slipsum.com/)

123
content/post/2014-09-23-publishing-to-pypi.md Normal file
View File

@@ -0,0 +1,123 @@
---
title: Publishing on PyPi
date: 2014-09-27 10:00:00
tags:
- github
- python
- pypi
---
Developing a python module and publishing it on Github is cool, but most
of the times you want others to download and use it easily. That is the
role of PyPi, the python package repository. In this post I show you how
to publish your package in less than 10 minutes.
Choose a fancy name
-------------------
If you haven't done so yet, take a minute or two to think about this.
To publish on PyPi you need a name for your package that isn't taken.
What's more, a catchy and unique name will help people remember your
module and feel more inclined to at least try it.
The package name should hint what your module does, but that's not
always the case. That's your call. I personally put uniqueness and
memorability over describing the functionality.
Create a .pypirc configuration file
-----------------------------------
```cfg
[distutils] # this tells distutils what package indexes you can push to
index-servers =
pypi # the live PyPI
pypitest # test PyPI
[pypi] # authentication details for live PyPI
repository = https://pypi.python.org/pypi
username = { your_username }
password = { your_password } # not necessary
[pypitest] # authentication details for test PyPI
repository = https://testpypi.python.org/pypi
username = { your_username }
```
As you can see, you need to register both in the [main pypi
repository](https://pypi.python.org/pypi?%3Aaction=register_form) and
the [testing
server](https://testpypi.python.org/pypi?%3Aaction=register_form). The
usernames and passwords might be different, that is up to you!
Prepare your package
--------------------
This should be the structure:
```
root-dir/ # Any name you want
setup.py
setup.cfg
LICENSE.txt
README.md
mypackage/
__init__.py
foo.py
bar.py
baz.py
```
### setup.cfg
```cfg
[metadata]
description-file = README.md
```
The markdown README is the *de facto* standard in Github, but you can
also use rST (reStructuredText), the standard in the python community.
### setup.py
```python
from distutils.core import setup
setup(name = 'mypackage',
packages = ['mypackage'], # this must be the same as the name above
version = '{ version }',
description = '{ description }',
author = '{ name }',
email = '{ email }',
url = 'https://github.com/{user}/{package}', # URL to the github repo
download_url = 'https://github.com/{user}/{repo}/tarball/{version}',
keywords = ['websockets', 'display', 'd3'], # list of keywords that represent your package
classifiers = [], )
```
You might notice that the download_url points to a Github URL. We could
host our package anywhere, but Github is a convenient option. To create
the tarball and the zip packages, you only need to tag a tag in your
repository and push it to github:
```bash
git tag {version} -m "{ Description of this tag/version}"
git push --tags origin master
```
Push to the testing/main pypi server
------------------------------------
It is advisable that you try your package on the test repository and fix
any problems first. The process is simple:
```shell
python setup.py register -r {pypitest/pypi} python setup.py sdist upload -r {pypitest/pypi}
```
If everything went as expected, you can now install your package through
pip and browse your package's page. For instance, check my senpy
package: <https://pypi.python.org/pypi/senpy>
```shell
pip install senpy
```

82
content/post/2014-10-09-proxies.md Normal file
View File

@@ -0,0 +1,82 @@
---
title: Proxies with Apache and python
date: 2014-10-09 10:00:00
tags:
- python
- apache
- proxy
- gunicorn
- uwsgi
---
This is a quick note on proxying a local python application (e.g. flask)
to a subdirectory in Apache. This assumes that the file wsgi.py contains
a WSGI application with the name *application*. Hence, wsgi:application.
Gunicorn
--------
```apache
<Location /myapp/>
ProxyPass http://127.0.0.1:8888/myapp/
ProxyPassReverse http://127.0.0.1:8888/myapp/
RequestHeader set SCRIPT_NAME "/myapp/"
</Location>
```
<!--more-->
**Important**: *SCRIPT_NAME* and the end of *ProxyPass* URL **MUST BE
THE SAME**. Otherwise, Gunicorn will fail miserably.
Try it with:
```bash
venv/bin/gunicorn -w 4 -b 127.0.0.1:8888 --log-file - --access-logfile - wsgi:application
```
UWSGI
-----
This is a very simple configuration. I will try to upload one with more
options for uwsgi (in a .ini file).
```apache
<Location /myapp/>
SetHandler uwsgi_handler
uWSGISocker 127.0.0.1:8888
</Location>
```
Try it with:
```bash
uwsgi --socket 127.0.0.1:8888 -w wsgi:application
```
### Extra: Supervisor
If everything went as expected, you can wrap your command in a
supervisor config file and let it handle the server for you.
```ini
[unix_http_server]
file=/tmp/myapp.sock ; path to your socket file
[supervisord]
logfile = %(here)s/logs/supervisor.log
childlogdir = %(here)s/logs/
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
logfile = %(here)s/logs/supervisorctl.log
serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL for a unix socket
[program:myapp]
command = venv/bin/gunicorn -w 4 -b 0.0.0.0:5000 --log-file %(here)s/logs/gunicorn.log --access-logfile - wsgi:application
directory = %(here)s
environment = PATH=%(here)s/venv/bin/
logfile = %(here)s/logs/myapp.log
```

109
content/post/2014-12-09-zotero.md Normal file
View File

@@ -0,0 +1,109 @@
---
title: Zotero
date: 2014-12-09 12:12:12
tags:
- zotero
- webdav
- nginx
- apache
---
[Zotero](https://www.zotero.org/) is an Open Source tool that lets you
organise your bibliography, syncing it with the cloud. Unlike other
alternatives such as [Mendeley](http://www.mendeley.com), Zotero can
upload the attachments and data to a private cloud via WebDav.
If you use nginx as your web server, know that even though it provides
partial support for webdav, Zotero needs more than that. Hence, you will
need another webdav server, and optionally let nginx proxy to it. This
short post provides the basics to get that set-up working under
Debian/Ubuntu.
Setting up Apache
-----------------
First we need to install Apache:
```bash
sudo apt-get install apache2
```
Change the head of "/etc/apache2/sites-enabled/000-default" to:
```apache
<VirtualHost *:880>
```
Then, create a file /etc/apache2/sites-available/webdav:
```apache
Alias /dav /home/webdav/dav
<Location /dav>
Dav on
Order Allow,Deny
Allow from all
Dav On
Options +Indexes
AuthType Basic
AuthName DAV
AuthBasicProvider file
AuthUserFile /home/webdav/.htpasswd
Require valid-user
</Location>
```
Ideally, you want your webdav folders to be private, adding
authentication to them. So you need to create the webdav and zotero
users and add the passwords to an htpasswd file. Even though you could
use a single user, since you will be configuring several clients with
your credentials I encourage you to create the zotero user as well. This
way you can always change the password for zotero without affecting any
other application using webdav.
```bash
sudo adduser webdav
sudo htpasswd -c /home/webdav/.htpasswd webdav
sudo htpasswd /home/webdav/.htpasswd zotero
sudo mkdir -p /home/webdav/dav/zotero
```
Enable the site and restart apache:
```bash
sudo a2enmod webdav
sudo a2enmod dav_fs
sudo a2ensite webdav
sudo service apache2 restart
```
At this point everything should be working at
<http://><your_host>:880/dav/zotero
Setting up NGINX
----------------
After the Apache side is working, we can use nginx as a proxy to get
cleaner URIs. In your desired site/location, add this:
```nginx
location /dav {
client_max_body_size 20M;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:880;
}
```
Now just reload nginx:
```bash
sudo service nginx force-reload
```
Extras
------
- [Zotero Reader](http://zoteroreader.com/) - HTML5 client
- [Zandy](https://github.com/ajlyon/zandy) - Android Open Source
client

36
content/post/2015-04-10-github-dotfiles.md Normal file
View File

@@ -0,0 +1,36 @@
---
title: Sharing dotfiles
date: 2015-04-10 17:47:00
tags:
- github
- git
- dotfiles
---
Today's post is half a quick note, half public shaming. In other words,
it is a reminder to be very careful with OAuth tokens and passwords.
As part of moving to emacs, I starting using the incredibly useful
[gh.el](https://github.com/defunkt/gist.el). When you first use it, the
extension saves either your password or an OAuth token in your
.gitconfig file. This is cool and convenient, unless you [happen to be
publishing your .gitconfig file in a public
repo](https://github.com/balkian/dotfiles).
So, how can you still share your gitconfig without sharing your
password/token with the rest of the world? Since Git 1.7.0, you can
[include other files in your
gitconfig](http://stackoverflow.com/questions/1557183/is-it-possible-to-include-a-file-in-your-gitconfig).
```config
[include]
path = ~/.gitconfig_secret
```
And now, in your .gitconfig_secret file, you just have to add this:
```config
[github]
user = balkian
token = "< Your secret token >"
```

20
content/post/2016-09-28-tqdm.md Normal file
View File

@@ -0,0 +1,20 @@
---
title: Progress bars in python
date: 2016-09-28 18:47:00
tags:
- python
---
[tqdm](https://github.com/noamraph/tqdm) is a nice way to add progress
bars in the command line or in a jupyter notebook.
![image](https://camo.githubusercontent.com/48838faaa8d00ea297f18e5bf55d3c6bb4e0ba6b/68747470733a2f2f692e696d6775722e636f6d2f686539417735432e676966)
```python
from tqdm import tqdm
import time
for i in tqdm(range(100)):
time.sleep(1)
```

104
content/post/2019-01-06-zigbee2mqtt.md Normal file
View File

@@ -0,0 +1,104 @@
---
title: Controlling Zigbee devices with MQTT
date: 2019-01-06 10:00:00
tags:
- mqtt
- iot
- zigbee
---
This is a short tutorial on connecting a zigbee device (an Aqara cube)
to an MQTT server, so you can control your zigbee devices from the
network.
If you're anything like me, you're probably a sucker for IoT devices.
For a long time, I've been using WiFi-enabled lights, and Amazon dash
buttons to control them. To keep these (cheap Chinese) internet enabled
devices away from your network and their respective cloud services,
you'll probably want to set up a dedicated network in your router (more
on this on a future post, maybe). Another disadvantage of WiFi devices
is that they're relatively power hungry.
A popular alternative is using ZigBee for communication. It is a
dedicated protocol similar to bluetooth (BLE), with lower power
requirements and bitrate.
Take the (super cute) aqara cube as an example. It is a small cube that
detects rotation on all of its axes, and tapping events. Here's a
video:
{{< youtube 5YtqG1wEnng >}}
To connect to zigbee devices you will need a zigbee enabled gateway
(a.k.a. hub), which connects to your WiFi network and your zigbee
devices. Once again, this means adding an internet-enabled device to
your home, and probably a couple of cloud services.
As an alternative, you can set up your own zigbee gateway, and control
it to your home automation platform of choice (e.g. home assistant). We
will cover how to set up a zigbee2mqtt gateway that is also connected to
an MQTT server, so you can use MQTT to control your devices and get
notifications.
What you need:
- [Aqara
cube](https://www.aliexpress.com/item/Original-Xiaomi-Mi-Aqara-Cube-Smart-Home-Controller-6-Action-Operation-Fr-Home-Device-Zigbee-Version/32892947622.html?spm=a2g0s.9042311.0.0.3da24c4dXV8sBI).
- [CC2531 zigbee
sniffer](https://www.aliexpress.com/item/Wireless-Zigbee-CC2531-CC2540-Zigbee-Sniffer-Bluetooth-BLE-4-0-Dongle-Capture-Module-USB-Programmer-Downloader/32907587711.html?spm=a2g0s.9042311.0.0.3da24c4dXV8sBI).
- [CC-debugger](https://www.aliexpress.com/item/CFSUNBIRD-CC-DEBUGGER-Debugger-and-Programmer-for-RF-System-on-Chips-TI-ORIGINAL-Fast-hipping/32813122315.html?spm=a2g0s.9042311.0.0.3da24c4dXV8sBI).
You will need to flash your sniffer. For that, you only need to follow
the instructions from the [zigbee2mqtt
documentation](https://koenkk.github.io/zigbee2mqtt/).
Once you're done flashing, you're ready to set up the zigbee2mqtt
server. For convenience, I wrote a simple docker-compose to deploy a
zigbee2mqtt server and a test mosquitto server:
```yaml
version: '2.1'
services:
zigbee2mqtt:
image: koenkk/zigbee2mqtt
container_name: zigbee2mqtt
restart: always
volumes:
- ./z2m-data/:/app/data/
devices:
- "/dev/ttyACM0"
networks:
- hass
mqtt:
image: eclipse-mosquitto
ports:
- 1883:1883
- 9001:9001
networks:
- hass
volumes:
- ./mosquitto.conf:/mosquitto/config/mosquitto.conf
networks:
hass:
driver: overlay
```
You can test your installation with:
{{<highlight shell "linenos=" >}}
❯ mosquitto_sub -h localhost -p 1883 -t 'zigbee2mqtt/#'
online
{"battery":17,"voltage":2925,"linkquality":149,"action":"rotate_right","angle":12.8}
{"battery":17,"voltage":2925,"linkquality":141,"action":"slide","side":2}
{"battery":17,"voltage":2925,"linkquality":120}
{"battery":17,"voltage":2925,"linkquality":141,"action":"wakeup"}
{{< / highlight >}}
zigbee2mqtt supports the following events for the aqara cube: shake,
wakeup, fall, tap, slide, flip180, flip90, rotate_left and
rotate_right. Every event has additional information, such as the sides
involved, or the degrees turned.
Now you are ready to set up home assistant support in zigbee2mqtt
following [this
guide](https://koenkk.github.io/zigbee2mqtt/integration/home_assistant.html).