Wednesday, April 10, 2019

Object Detection With Deep Neural Networks

Recently I red a series of papers about object detection using deep neural networks. Here is summary of the reading.

R-CNN: Region-based Convolutional Networks for Accurate Object Detection and Segmentation
用传统方法提出region proposal,train一个classifier和一个location regressor。classifier为了提高精度先用softmax train,然后fc层提出来的feature用svm fit,regressor单独作为一个network

用到了selective search生成proposal,从一个over-segmentation开始repeatedly merge similar regions. 然后每个region用传统的descriptor提feature,用bag-of-words model+ svm分类

Fast R-CNN

Faster R-CNN
提出一个region proposal network (RPN) 共享feature extraction的networks,不增加计算开销的情况下把上一版最耗时的region proposal步骤变成自动从network生成。

[Linux Tips] Bash Shell (I) declare a new variable:
2. in linux, before you run you shell-script, you need to change the mod of script file.
chmod 755 script_name
(This will change the mode of file to owner: rwx, group and others: r-x)
maybe ./script_name is more precisely. This command will add the permission of execution to this script file.
3. use a variable:
4. to do something on all certain files under current directory:
for file in ` ls *[^smp].byu `
# do something on every file $file
echo $file

if ...
fi # end of if :)

5. substring:
To take part of a string:
echo subtring
result: es
The string starting position is zero-based.

6. Searching and Replacing Substrings within Strings:
In this method you can replace one or more instances of a string with another string. Here is the basic syntax:
alpha="This is a test string in which the word \"test\" is replaced."
The string "beta" now contains an edited version of the original string in which the first case of the word "test" has been replaced by "replace". To replace all cases, not just the first, use this syntax:

Note the double "//" symbol.

Here is an example in which we replace one string with another in a multi-line block of text:

list="cricket frog cat dog"
poem="I wanna be a x\n\
A x is what I'd love to be\n\
If I became a x\n\
How happy I would be.\n"
for critter in $list; do
echo -e ${poem//x/$critter}
Run this example:
$ ./
I wanna be a cricket
A cricket is what I'd love to be
If I became a cricket
How happy I would be.
I wanna be a frog
A frog is what I'd love to be
If I became a frog
How happy I would be.
I wanna be a cat
A cat is what I'd love to be
If I became a cat
How happy I would be.
I wanna be a dog
A dog is what I'd love to be
If I became a dog
How happy I would be.
Silly example, huh? It should be obvious that this search & replace capability could have many more useful purposes.

7. to run matlab (e.g plot some graph) without start matlab window
matlab -nodesktop -nosplash -r "plot_data;quit;"
#here we have a .m file named "plot_data.m" under current dir

Tuesday, April 2, 2019

[Machine Learning] from distance to kernel (classification via SVM)

Since for many machine learning techniques, you can use kernel trick. This is important to find a kernel, or explicitly construct a kernel from your data. One possible way to do this is calculate metric distance.
This article only gives some fundamental ideas for this transformation, without any strict provment.

Once you have a matrix of distance to measure the difference or distance between a pair of instances, denote I_i, I_j. You may define a kernel function /rho (I_i, I_j) = f(d(I_i,I_j)), where d(I_i,I_j) is the distance you've obtained. The most common choice is natural exponential.

Another webpage to list some ways to calculate kernel from original feature space:

For the requirement for positive kernel, you may refer to this paper:

Monday, April 1, 2019

[Linux Tips] Screen, date

[Linux Tips] includes a series of very common tips for development in Linux environment. By default, I am using Ubuntu.

This is used to create/maintain a session. This is very useful when you ssh to a remote machine and running some jobs in a interactive mode. Imagine if your network is disconnected, the thread to run your job will be killed. Screen can help you to create a session. You can attach (enter) or detach (quit) from the session.
1. To create a session
screen -S session_name
or simply,

2. To list all sessions available.
screen -ls

3. Attach a session
screen -r session_name (a prefix is enough)

4. Detach a session

5. Delete a session
screen -X -S session_name kill

When you run some bash script. It's common that you want to name your output with current date/time.
In bash shell script you can do it as below.
dt_suffix=`date +'%d%m%H%M'`
Basically, it just assigns the date/time to a variable. And the string comes from date command. The +"%d%m%H%M" is the optional format parameter.

Bash Shell script, some common usages.
For-loop with an array.

declare -a your_array=(
for ((i = 0; i< ${array_len}; i++));
echo "processing" ${your_array[${i}]}
# Do something here.

Check whether directory or file already exists.
if [ ! -f ${file_path} ]; then
echo "File ${file_path} does not exist!"
echo "File ${file_path} already exists!"

if [ ! -d ${dir_path} ]; then
echo "Directory ${dir_path} does not exist!"
echo "Directory ${dir_path} already exists!"

Saturday, November 4, 2017

Andrew Ng的AI新课程

Andrew Ng离开baidu后三件大事之一是成立,开设五门深度学习课程。网易云课堂搬运并翻译了前三门课。免费听一下,感觉课程内容虽然比较浅显偏engineering多一些,但总体还是质量颇高,收获不少,按照教学大纲总结如下 (慢慢更新)。

第一周  深度学习概论:


1.1  欢迎来到深度学习工程师微专业
1.2  什么是神经网络?
介绍神经网络,house price prediction的例子,因为price不能为负所以曲线变成ReLU,很有意思的一个引出ReLU的方式,hiden unit 也称之为neuron。neural network就是stack neuron (like Lego brick) together toform a network.很直观。
1.3  用神经网络进行监督学习
几乎所有有价值的机器学习/神经网络技术都是supervised learning。也就是model一个x到y的映射函数。相对应的unsupervised learning,不存在y,只能指望data can tell something itself。
house price prediction用standard NN解决;image understanding,object detection这类问题用convolutional NN (CNN)解决;time series,or temporal sequence数据问题用Recurrent NN (RNN)解决;
structural data:房屋size等;unstructual data: audio/image/text, etc.

1.4  为什么深度学习会兴起?
举例:ReLU代替sigmoid。主要的优势在于sigmoid在两侧gradient几乎为0,导致gradient decent优化速度变慢。
interesting graph: idea ->code->experiments->idea... computation的提高导致这个iterative process加速,能带来更多更好结果。

1.5  关于这门课
1.6  课程资源
鼓励上forum讨论问题。其他问题也可以联系。可以看出这个公司/机构的目的是培训为主,所以提到如果有公司需要培训hundreds of empolyees with deep learning expertise,可以联系他们,大学老师想开deep learning课的也可联系他们。

第二周  神经网络基础:


2.1  二分分类
2.2  logistic 回归
2.3  logistic 回归损失函数
2.4  梯度下降法
2.5  导数
2.6  更多导数的例子
2.7  计算图
2.8  计算图的导数计算
2.9  logistic 回归中的梯度下降法
2.10  m 个样本的梯度下降
2.11  向量化
2.12  向量化的更多例子
2.13  向量化 logistic 回归
2.14  向量化 logistic 回归的梯度输出
2.15  Python 中的广播
2.16  关于 python / numpy 向量的说明
2.17  Jupyter / Ipython 笔记本的快速指南
2.18  (选修)logistic 损失函数的解释

第三周  浅层神经网络:


3.1  神经网络概览
3.2  神经网络表示
3.3  计算神经网络的输出
3.4  多样本向量化
3.5  向量化实现的解释
3.6  激活函数
3.7  为什么需要非线性激活函数?
3.8  激活函数的导数
3.9  神经网络的梯度下降法
3.10  (选修)直观理解反向传播
3.11  随机初始化

第四周  深层神经网络:


4.1  深层神经网络
4.2  深层网络中的前向传播
4.3  核对矩阵的维数
4.4  为什么使用深层表示
4.5  搭建深层神经网络块
4.6  前向和反向传播
4.7  参数 VS 超参数
4.8  这和大脑有什么关系?

Monday, June 30, 2014

How does Matlab calculate the eccentricity of a region

In matlab, there is a built-in function to calculate properties of a region.

And as said in help message:
'Eccentricity' — Scalar that specifies the eccentricity of the ellipse that has the same second-moments as the region. The eccentricity is the ratio of the distance between the foci of the ellipse and its major axis length. The value is between 0 and 1. (0 and 1 are degenerate cases; an ellipse whose eccentricity is 0 is actually a circle, while an ellipse whose eccentricity is 1 is a line segment.) This property is supported only for 2-D input label matrices.

So, the idea is to fit using a ellipse with same second-moments as the region.
What does it mean?
The answer is in this thread:

To simplify, the idea is to calculate the co-variance matrix, then do eign-value decomposition. Eigen-values are those axis length, minor and major one. While eigen-vectors are the directions of minor/major axis.

length of major axis = 2a, minor axis = 2b, then the foci = c, then:
eccentricity E = c/a = sqrt(1-(b/a)^2)
a^2-b^2 = c^2.