Stochastic Approximation – Beginning


Hi everyone, this time I would like to present you something more ambitious than previous blog posts.

Since it is my last semester at Wroclaw University of Technology, the time of BSc project has come. I will write down a mini-series about my preparation, sometimes it can be more technical or just theory. This write is a introduction what is about my project.


In the middle of may 2014 I have realized that it’s time to choose a subject for my final project, I had plenty of ideas my like : own interpreter, nosql database, some new tools, implement continuous delivery  but I wanted also to do something not ordinary so after few talks with my supervisor we have agreed to parallel implementation of some optimization algorithm, then we chose Global Stochastic Optimization algorithm described by Sid Yakowitz.

Optimization ?!

Yes, you may wonder that it’s something boring but it’s not!

Especially, this field of studies is getting more popular or at least should be, because we can use those high specialized algorithms on our big data sets to optimize or find solution of our problems.

Where it’s useful ?

We might use it in problems where we want to optimize our problem. I would say that the problem of optimization is everywhere from signal processing to production in some sort of industry. Almost everything can be represented, described as a mathematical model and then optimize.

As you already know that this algorithm can solve some optimization problems you may ask so how many different information (variables) Can I provide ? Basically this algorithm is powerful and quick up to 9th dimensional functions

 What’s my plan

My plan for the next couple of weeks is to create fully working algorithm for 3D problems, so far I have read one article SIAM and one section of the book, it took quite long time to understood what is written (a lot of math!).

First prototype written in R is almost done (but about this I will type separate blog post)


First of all chapter of this book (Random search under additive noise) is a basic knowledge then you can read this SIAM which is less difficult than this book.

Repositories :