Dangerous Java

Git Bash Colored Branch Name On Mac Os X

2013-11-06T00:00:00-08:00

I have moved recently to Mac OS X (10.8.2) and I am still using git-svn to work with our corporate repository with git. First of all I wanted to add git bash completion:

 curl https://raw.github.com/git/git/master/contrib/completion/git-completion.bash -o ~/.git-completion.bash
 echo "source ~/.git-completion.bash" >> ~/.bash_profile"

Then I wanted to beautify command prompt and put this line to ~/.profile:

 export PS1="\[\033[36m\]\u\[\033[m\]@\[\033[32m\]\h:\[\033[33;1m\]\w\[\033[m\] \$ "

The next step was to add git branch name to the command prompt.

 curl https://github.com/git/git/blob/master/contrib/completion/git-prompt.sh -o ~/.git-prompt.bash

Then put following line to the beginning of ~/.profile

 source ~/.git-prompt.bash

Next we need to modify the PS1 variable.

 export PS1="\[\033[36m\]\u\[\033[m\]@\[\033[32m\]\h:\[\033[33;1m\]\w\[\033[m\]\e[0;32m$(__git_ps1 " (%s)")\e[m \$ "

This will work, but only once! It evaluates expression for only one time. After googling around I have found that one of possible solutions is to escapte dollar sign (’$’) at the beginning of __git_ps1 invocation. However, this doesn’t solve problem and breaks PS1 at all. The solution is to use single quote characters to define string for PS1 variable. This tells bash to evaluate expression each time.

The working example:

 export PS1='\[\033[36m\]\u\[\033[m\]@\[\033[32m\]\h:\[\033[33;1m\]\w\[\033[m\]\e[0;32m$(__git_ps1 " (%s)")\e[m \$ '

I haven’t found it in google, hope someone finds it helpful.

Git Svn Command Error 128

2013-06-27T00:00:00-07:00

On my current work we’re using SVN as the code repository for our main project and all sub projects. About a month ago I started working with this repo via git-svn. So I have my local git repository, commit there changes and then push them to the SVN repository. I will not describe how to do that, maybe as a separate post.

My openSUSE comes with pretty old 1.7.3.4 version of git. However, it was good until today. I got following error message after trying to fetch changes:

> git svn fetch --fetch-all 
  Index mismatch: 1bac9c8667e2b3772d71108562404c95c8f7d9b0 != 97bb5a8fb29d0cb3a9b38456bf73d3c126d7c6fd
  rereading 0bf8f076838b6078e7a8bf68b0f2a234f509cf46
    M	src/com/company/project/package/UpdatedClass.java
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and acd2afc03bb5b2dc10126905cfd0cb2774aa4b2b. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and 99a066db650463dd8c3c926129bf462edeb2ed14. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and 5aa2ecdf0f8fd56480f5d71e67c2d190d1ebcbc0. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and dd5cd8aaf1d01ce8de212c29faa10891e8b45578. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and 2537435f7970943e4cb8f15a1864c641bd09a231. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and bb13cc7c762e2589966a459c8f7a6a332d7939cb. Ignoring merge info.
  W: Cannot find common ancestor between 0bf8f076838b6078e7a8bf68b0f2a234f509cf46 and 0ca1856db48e52c300f6a0a04beb54e9f5edc747. Ignoring merge info.
  fatal: ambiguous argument '618f50e5893d13f274652b0dd87b29b17cd32072^..c53f7450af7b937ca9e6d9659d9c7e7467b780ca': unknown revision or path not in the working tree.
  Use '--' to separate paths from revisions
  rev-list 618f50e5893d13f274652b0dd87b29b17cd32072^..c53f7450af7b937ca9e6d9659d9c7e7467b780ca: command returned error: 128

The only possible solution I’ve found is to remove git-svn caches:

rm .git/svn/.caches/check_cherry_pick.db
rm .git/svn/.caches/lookup_svn_merge.db

Then do the same fetch. However, it looks like my version of git is so old that even this doesn’t help.

To solve this I managed to update git to the latest version by cloning official repo at git clone https://github.com/git/git.git . Then I’ve uninstalled currentl git package, compiled sources, cleared SVN cache again and voila! Everything works again.

I’ve thought that maybe someone would find this information useful.

Raspberry Pi Io

2013-04-05T00:00:00-07:00

Recently I got three Raspberry PIs. Ignoring fails of other PI users I want to set up a simple NAS on tiny PC. So, first all I’ve thought about storing data on SD card to increase I/O speed. However, some day this will kill both the SD card and as a result entire linux system. I got an external WD drive, which was bought to store wedding video archive. After re-partitioning I got a new 160GB ext4.

Checking raspberry PI write to external USB drive, ext4:

# dd if=/dev/zero bs=512K count=2048 of=/media/disk-linux/speed.tmp
1073741824 bytes (1.1 GB) copied, 43.4 s, 24.7 MB/s

# dd if=/dev/zero bs=1M count=1024 of=/media/disk-linux/speed.tmp
1073741824 bytes (1.1 GB) copied, 42.7062 s, 25.1 MB/s

# dd if=/dev/zero bs=16M count=64 of=/media/disk-linux/speed.tmp
1073741824 bytes (1.1 GB) copied, 46.4082 s, 23.1 MB/s

Checking read from same ext4 external USB drive:

# dd if=/media/disk-linux/speed.tmp bs=512k count=2048 of=/dev/null
1073741824 bytes (1.1 GB) copied, 38.1892 s, 28.1 MB/s

# dd if=/media/disk-linux/speed.tmp bs=1M of=/dev/null
1073741824 bytes (1.1 GB) copied, 38.2423 s, 28.1 MB/s

# dd if=/media/disk-linux/speed.tmp bs=16M of=/dev/null
1073741824 bytes (1.1 GB) copied, 38.3897 s, 28.0 MB/s

Similar tests for SD card:

# dd if=/dev/zero bs=512K count=2048 of=/usr/data/speed.tmp
1073741824 bytes (1.1 GB) copied, 62.1034 s, 17.3 MB/s

# dd if=/dev/zero bs=1M count=1024 of=/usr/data/speed.tmp
1073741824 bytes (1.1 GB) copied, 64.2532 s, 16.7 MB/s

# sudo dd if=/usr/data/speed.tmp bs=512k of=/dev/null
1073741824 bytes (1.1 GB) copied, 50.2513 s, 21.4 MB/s

Conclusion: just use the external drive to store data on raspberry PI. Don’t forget to buy good power source for both raspberry PI and external drive. (I use USB hub with separate power adapter)

Simple Progress Bar In Bash

2012-11-25T00:00:00-08:00

Bash coding is a mistreious thing I can’t get used at. The syntax is awful, it doesn’t look like anything that I use in my every day work.

Today, I’m doing an upgrade of our support desk tool. Their upgrade tool makes a mandatory backup. It’s size on our installation in ~46GB, so when waiting I’ve decided to write an shell script that will check an upgrade file to reach the pre-defined size, display progress bar and size in MB.

The following things I’ve learned during writing:

to read the output of command right to the variable - use back quotes `. The key is usually at the top lef corner of the main keyboard right below the ‘Esc’ button. I almost haven’t used this button before, so this is like a new discovery for me :)
arithmetic operations in shell go in double brackets and should be preceeded by the $ symbol, like:
```
a=1
$b = $(( a + 1))
```
the for cycle may be defined either as an quick sequence
```
for i in {0..5}
```

I couldn’t use the variables here neither for the lower bound nor for the upper, so given up with it. The second style is an ‘arithmetic’ style:

	for (( expr1; expr2; expr3 ))

The variables may be defined in a java (or c++) friendly way, so that’s my choice.

To display progress bar the easiest (and maybe the only) way is to rewrite the same line by prining the caret return symbol each time at the end of progress bar.

Here is the code that I’ve got (also available at github):

#!/bin/bash
to_monitor=$1
target_gb=$2
read_mb=0
while [ $read_mb -le $(( $target_gb*1024 )) ]
do
        read_mb=`du -sm $to_monitor | cut -f 1`
        read_gb=$(( $read_mb/1024 ))
        echo -ne '[ '
        for (( gb=0; gb<$target_gb; gb++ ))
            	do
                    	if [ $read_gb -ge $gb ]
                    	then
                                echo -ne '#';
                    	else
                                echo -ne ' '
                    	fi
            	done
    	perc=$(( $read_mb*100/($target_gb*1024) ))
    	echo -ne " $perc%] $read_mb of $(($target_gb*1024))\r"
    	sleep 1
done

Opensuse Yguard Issue

2012-10-24T00:00:00-07:00

Recently we’ve installed a new headless PC with openSUSE 12.2 to run tests. We’re testing GUI using FEST Swing Library and for running each parallel set of tests in isolation we use jenkins Xvnc plugin. It starts up an Xvnc server before every build and tears it down after build is finished. Each time it looks for a free display number, then sets DISPLAY environment variable.

The problem that we’ve faced was during obfuscation when Xvnc server is started and DISPLAY variable is set:

/opt/hudson/workspace/Testing MySQL/dev_frozen/build.xml:705: The following error occurred while executing this line:
/opt/hudson/workspace/Testing MySQL/dev_frozen/build.xml:429: java.lang.NoClassDefFoundError: java.awt.Container
	at java.lang.Class.getDeclaredMethods0(Native Method)
	at java.lang.Class.privateGetDeclaredMethods(Class.java:2427)
	at java.lang.Class.privateGetPublicMethods(Class.java:2547)
	at java.lang.Class.getMethods(Class.java:1410)
	at com.yworks.yguard.obf.Cl.scanExtSupers(Unknown Source)
	at com.yworks.yguard.obf.Cl.scanExtSupers(Unknown Source)
	at com.yworks.yguard.obf.Cl.scanNameSpaceExcept(Unknown Source)
	at com.yworks.yguard.obf.Cl.resolveOptimally(Unknown Source)
	at com.yworks.yguard.obf.ClassTree$4.classAction(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.walkTree(Unknown Source)
	at com.yworks.yguard.obf.ClassTree.resolveClasses(Unknown Source)
	at com.yworks.yguard.obf.GuardDB.createMap(Unknown Source)
	at com.yworks.yguard.obf.GuardDB.remapTo(Unknown Source)
	at com.yworks.yguard.ObfuscatorTask.execute(Unknown Source)
	at com.yworks.yguard.YGuardTask.execute(Unknown Source)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:390)
	at org.apache.tools.ant.Target.performTasks(Target.java:411)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
	at org.apache.tools.ant.helper.SingleCheckExecutor.executeTargets(SingleCheckExecutor.java:38)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
	at org.apache.tools.ant.taskdefs.Ant.execute(Ant.java:442)
	at org.apache.tools.ant.taskdefs.CallTarget.execute(CallTarget.java:105)
	at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:291)
	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
	at org.apache.tools.ant.Task.perform(Task.java:348)
	at org.apache.tools.ant.Target.execute(Target.java:390)
	at org.apache.tools.ant.Target.performTasks(Target.java:411)
	at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1397)
	at org.apache.tools.ant.Project.executeTarget(Project.java:1366)
	at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
	at org.apache.tools.ant.Project.executeTargets(Project.java:1249)
	at org.apache.tools.ant.Main.runBuild(Main.java:801)
	at org.apache.tools.ant.Main.startAnt(Main.java:218)
	at org.apache.tools.ant.launch.Launcher.run(Launcher.java:280)
	at org.apache.tools.ant.launch.Launcher.main(Launcher.java:109)

It tooks couple of hours for me to find out that the reason for this thing is Xvnc plugin, which sets up DISPLAY variable and forces JVM to run in headful mode. I’ve guessed that class loader fails to load some library in such case.

Then I’ve decided to run following test sample after running Xvnc:

import javax.swing.*;

public class FrameTest extends JFrame
{
    public static void main(String[] args)
    {
        FrameTest f = new FrameTest();
        f.setTitle("Test");
        f.setBounds(0, 0, 100, 100);
        f.setDefaultCloseOperation(WindowConstants.EXIT_ON_CLOSE);
        f.setVisible(true);
    }
}

This results in following error:

Exception in thread "main" java.lang.UnsatisfiedLinkError: /usr/lib64/jvm/jdk1.6.0_35/jre/lib/amd64/xawt/libmawt.so: libXtst.so.6: cannot open shared object file: No such file or directory
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1807)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1703)
	at java.lang.Runtime.load0(Runtime.java:770)
	at java.lang.System.load(System.java:1003)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1807)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1724)
	at java.lang.Runtime.loadLibrary0(Runtime.java:823)
	at java.lang.System.loadLibrary(System.java:1028)
	at sun.security.action.LoadLibraryAction.run(LoadLibraryAction.java:50)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.awt.NativeLibLoader.loadLibraries(NativeLibLoader.java:38)
	at sun.awt.DebugHelper.<clinit>(DebugHelper.java:29)
	at java.awt.Component.<clinit>(Component.java:566)
Could not find the main class: FrameTest.  Program will exit.

After googling around the solution came:

zypper in libXtst-devel libXtst6

Threadlocal Pain

2012-09-13T00:00:00-07:00

In our application we use several session factories, one per each database connection. Sessions are always created by thread local current-session context (we use default org.hibernate.context.ThreadLocalSessionContext implementation). The transactions are managed manually. If the transaction was opened but didn’t closed then that particular thread will have a session in the thread-local variable. When session factory gets closed the reference remains in that thread local, which produces a memory leak.

As for now I’ve found the only hardcore solution for such issue. So when session factory gets closed walk through all thread locals and clean up.

Intellij Idea Svn Sucks

2012-09-06T00:00:00-07:00

It just can’t neither merge nor load history of our repository. This really sucks. The only thing that delays my full movement to that IDE. So my way is now to use the eclipse to merge branches and see history.

Npe With Empty Stack Trace

2012-09-05T00:00:00-07:00

Every day gives the more pleasant problems. This time it’s the magic of the -XX:-OmitStackTraceInFastThrow VM flag. I don’t know, maybe someone faced problem when JIT decides that exception is thrown too often in same place and replaces it with empty stacktrace-less stub in all next occurrencies. I believe that kind of optimization was directed to leverage costs of stack trace reterieval.

However, in our case the exception fellt to infrequent:

2012-08-22 14:29:52,911 FATAL: java.lang.NullPointerException
2012-08-22 14:30:01,578 FATAL: java.lang.NullPointerException
2012-08-22 14:30:10,303 FATAL: java.lang.NullPointerException

Then JIT starts replacing NPE object with static stub for a week (!) of application worktime. Hell yeah! You rock, JIT!