I found the solution. Problem was the process / processes that were haunting in the "memory". Killing those darn ghost processes and rebooting after that with mouse button trick helped me out.
Just show the processes and find the process / processes that are causing the problem (even after power off).
Code:
ps -ef
sudo kill <process number>
After that reboot the computer and press the left mouse button down during early startup. I have wireless USB keyboard (not bluetooth) and it worked ok. Hopefully this helps if someone faces similar challenges.