April 22, 2017

AI Product improvement cycle

Artificial Intelligence (AI) & Machine Learning(ML) has significant advantage over traditional algorithms which we used to improve our products. This is mainly due to the fact that ML improves with big data unlike the traditional algorithms.




Big data is the key for the ML & AI. It helps ML learns and improve the accuracy. This is the main reason, companies such as Google benefits vastly from ML & AI. Moreover, AI & ML can improve the product significantly. When the product quality improves more users tend to use it and this generates more data. More data can be used for ML & AI to improve the product hence an infinite loop of improving the product. This is what I called -- “AI Product improvement cycle”



AI & ML has become an essential part of any product development. Therefore, analysing and implementing ML algorithms will improve the quality of the product which can leads to more users.

October 20, 2016

AWS.Net - Simple library for AWS services. i.e. S3, SQS & SES

Have worked on many AWS projects but found their library wasn't designed for simplicity. So thought of writing my own library for my pet projects with AWS. It's very simple and easy to integrate S3, SQS & SES.

Visit my Github project for more details on how to use it.  - https://github.com/ludmal/AWS.Net

For example, sending a message to the SQS is easy as this. 
var service = new SqsService<EmailMessage>(new AwsCredentials
{
  RegionEndpoint = RegionEndpoint.USWest2
});

service.QueueUrl = ConfigurationManager.AppSettings["EmailQueue"];

var response = service.Push(new HelloEmail());

And retrieving the message with Generic object is very simple.
var service = new SqsService<EmailMessage>(new AwsCredentials
{
 //SQS Service Region
 RegionEndpoint = RegionEndpoint.USWest2
});

//SQS Queue Url
service.QueueUrl = ConfigurationManager.AppSettings["EmailQueue"];

var items = service.Process();

April 22, 2016

Chalk

Chalk is a simple library written by me to output console text with different colors. Enough said! Get the nuget package and start using it.

Install-Package Chalk

How to use:
Chalk.Blue("Blue")
Chalk.Red("Red")
Chalk.Green("Green")
Chalk.Gray("Gray")
Chalk.Yellow("Yellow")

Here is the Github link with Examples.

May 29, 2015

Better AngularJs service for SignalR


SignalR makes our lives easy when buidling realtime webapps. Needless to say AngularJs has already found a permanant role in our webapps. Here is a quick AngularJs service which I wrote to integrate SignalR broadcasts.


 var app = angular.module('signalr', []);  
 app.factory('SignalRService', ['$rootScope', '$window', function ($rootScope, $window) {  
   var srv = {};  
   srv.init = function (hub) {  
     var connection = $.hubConnection();  
     var proxy = connection.createHubProxy(hub);  
     proxy.on('broadcast', function (jobs) {  
       angular.forEach(jobs, function (item) {  
         console.log(item.EventName, item.Data);  
         $rootScope.$broadcast(item.EventName, item.Data);  
       });  
     });  
     connection.start()  
      .done(function () {  
        console.log('Connected');  
      })  
      .fail(function () { console.log('Failed to connect!'); });  
   };  
   return srv;  
 }]);  

August 12, 2014

Future of Web Development : AnuglarJs + REST API ?

It's a paradigm shift in web development. Unlike the days we used to develop web apps using classic Asp, Asp.Net or Php, the focus has changed into decoupling the frontend and backend. More and more web apps are adapting this methodology mainly due the evolution of mobile usage.

With mobile & tablets revolution, developers are constantly finding ways to develop a single app to support all of the different devices and their browser sizes. It has been always difficult and costly to develop native apps for various different mobile platforms. Although the responsive design was the solution, it is extremely hard to avoid unnecessary page reloads while using the web app on mobile devices.

SPA - Single Page App concept was the solution to the problem. However until the release of AngularJs, it wasn't very enjoyable development for the developers. There were plethora of MVC JavaScript libraries but none of them was even nearly good as AngularJs. While the AnugularJs solves the frontend problem, the backend lies in RESTful API's. The main advantage in this architecture is the app is decoupled hence frontend and backend can be developed using different technologies. For example, while the frontend is AngularJs, the backend API can be developed in Node.js, Asp.Net Web API or any other API provider technology.

The basics of AngularJs + API driven Web app.


This should be the preferred approach for any new web application development. However for the apps which are monolith or legacy could still gain the benefits by modularizing the APIs as shown in the following example.  For example, some parts of the apps can be decoupled using an API layer on top of the legacy/monolith web application.


One of the best approaches to write a scalable backend API is by using micro-services architecture, which will be discussed in my next post.

Writing scalable & high performance web applications have been a challenge, by making it device agnostic is even challenging. It is evident that AngujarJs + APIs solve the challenging nature of web apps to support multiple devices and increase the performance. 



February 11, 2014

Sending Emails using Templates

Here is a simple Python module to send emails using HTML/TEXT templates. Feel free to use the code.

https://github.com/ludmal/pylib

Step 1:

Create a TEXT/HTML file with the Keys as below
Hi [username],

Thank you for your registration.

Best regards,
[from]
[url]

Step 2:

Create a template passing the values to replace with the keys in template
values = {}
values['username'] = 'Ludmal de silva!'
values['from'] = 'The Team'
values['url'] = 'http://www.ludmal.com'
temp = EmailTemplate(template_name='welcome.txt', values=values)

Step 3:

Create a Mail Server
server = MailServer(server_name='smtp.gmail.com', username='', password='', port=0,   require_starttls=True)

Step 4:

Create a mail message and send the email
msg = MailMessage(from_email='ludmal@gmail.com', to_emails=['ludmal@gmail.com'], subject='Welcome')
send(mail_msg=msg, mail_server=server, template=temp)
Follow me on Twitter @ludmal

February 7, 2014

Python File Archiver

I always wanted to quickly cleanup my harddrive without manually searching files and moving them. So here is the solution, a simple File Arhicver python script which I wrote to clean up my hard drive. It will search for files older than a given time period & size (parameterized) and then move to a Arhicve folder. I'll try to improve the code when I get some free time and here is the script file, download and feel free to use it.

https://github.com/ludmal/pylib/blob/master/file_archiver

November 27, 2013

tf-idf, Term Frequency–Inverse Document Frequency

tf-idf is stands for Term Frequency - Inverse document frequency and it is one of the effective algorithms to extract the keywords from a given document. It is often used in NLP and IR. The extraction is performed in a statistical measure by calculating the weight of a word in a document list or a corpus. Many search engines using the benefits of tf-idf to extract keywords, frequently with combinations of other algorithms. Word's weight or importance is measured by the word frequency in a document and offset by the same word's frequency in the given corpus.

tf-idf = term frequency x inverse document frequency

For example, if the word 'peace' appears 6 times in a document with 100 words the tf is 6/100 = 0.06. And if the corpus or document list contains 1000 documents and if 'peace' appears in 200 documents in the corpus then the idf is 1000/200 = 5. Hence the tf-idf for the word is 0.06 x 5 = 0.3. The weight or tf-idf is directly related to the importance word, i.e. if the tf-idf is higher then the importance of the word is high.



I've written a python module to extract the keywords from a given corpus. This is useful if you want to extract the keywords from a given website links and categorized them according to the keywords. You can use the code freely by downloading from the following Github location.

https://github.com/ludmal/pylib

Complete sample of the usage can be found here: https://github.com/ludmal/pylib/blob/master/sample.py



July 23, 2013

GIT Tortoise

I've recently moved all my projects to GIThub.com This is mainly due to the fact the popularity gain by GITHub compare to Google code. I was a big fan of Tortoise SVN and luckily it is now available for GIT as well. You can download it from the following location:
https://code.google.com/p/tortoisegit/

Also if anyone wants to use GIT with Visual Studio, here is how;

1. Install GIT for windows
https://code.google.com/p/msysgit/downloads/list?q=full+installer+official+git

2. Install GIT source control provider
http://gitscc.codeplex.com/

and your done! Don't forget to select GIT from the VS Source control settings.