JavaRebel supports Guice now
Good news:
“Google Guice Plugin 1.0 M1. Supports discovering new Google Guice (implicit) components, adding/removing implicit setter and field dependencies and reconfiguring @Singleton’s.”
Programming is hard by Stephan Schmidt
Good news:
“Google Guice Plugin 1.0 M1. Supports discovering new Google Guice (implicit) components, adding/removing implicit setter and field dependencies and reconfiguring @Singleton’s.”
Sorry for the inconvenience, got slashdotted by reddit. Never thought so many were
interested. And no, scaling was not one of the reasons the startup failed ;-) No ssh from my work so it took some time to fix it. Thanks for coming (back)
During the dot com boom I founded a software startup with some friends - with me as the CTO. We developed a software for knowledge management. It was a combination of blogs, wikis, a document management system, link managment, skill managment and more. We started in 1999 which was quite early for wikis and blogs (Moveable Type was announced 2001). The link management system was essentialy the same as Delicious later. Beside all those new ideas (for 1999 at least) there were three great features:
We’ve got quite some money as a seed investment from a VC and were quite happy and successfully developing our application. We did show it to many users and received very favorable feedback from big companies. Why did the startup fail and I’m no millionaire? There are a myriad of reasons, but as I wrote in “Rules for a successful business”, the rules for a sucessful business are easy:
- The customer is the most important thing in your business
- The best business plan is to sell people the things they want
- Your business is successful if your earnings are higher than your spendings
So the most important thing is to sell - a fact lots of startups forget. And we did too. After much thought it comes down to these six reasons why we failed (beside the obvious one that the VC market imploded when we needed money and noone was able to get any funding):
In more detail:
We didn’t sell anything because we didn’t have a product to sell. As good engineers we wanted to wait until the product is finished and then start selling. Midway we started selling nevertheless with a nearly finished 1.0. This led to too much focus on development and not enough on sales. Without a finished product we thought we couldn’t go to customers and try to sell it. We’ve slowly learned two things:
We didn’t sell anything because we had no sales person. Bummer. Of course we were looking for one and the business plan said: High priority is finding a sales person for the founding team. This took time and resources and we didn’t have that. If you want to sell something, get a sales founder or hire someone from the start.
We didn’t sell something because customers wouldn’t buy. The product was great, the customers favorable, but they took too long to decide. We wanted to sell a knowledge managment tool bottom up to project managers and through them to companies. But everytime a superior heard of knowledge managment he decided it should be on his agenda. So the topic of knowledge managment moved up the chain of command and we had no real decision maker. We talked to irrelevant people (because the topic became strategic fast for our customers) and lost lots of time. Ask people if they are allowed to buy your product. Get to the one who says “Yes” fast. We had several big companies in our sales queue and I’m sure they would have bought in the end, but as a startup we couldn’t wait. SAP for example compared to us could have waited and sold the product after 12 months. Selling enterprise software takes a lot of time.
The market window was not yet open. Noone heard of blogs, wikis and tagging. We had to educate our customers on the benefits of wikis (Everyone can edit! Everyone! How dare they!) and blogs (Everyone can have an opinion and write about it! Everyone!) and tagging (They can build ontologies! We need a comittee to define an ontology for everyone! Otherwise chaos will eat us!) Several years later it would have been much easier to sell a blog, wiki and tagging plattform.
All the founders were interested in technology. We’ve worked with EJBs (not mature back then), we made everything spit out XML and rendered that to HTML with XSLT (not fast enough), wrote our own OR-Mapper - what a stupid idea (Hibernate not available), tried CSS driven websites (not enough knowledge available back then). This lead to rewrites and took lots of our time. Discussions about technology - we should have talked about customers - took time too and led to frustration.
Plain and simple: we had the wrong business model. Selling software can eventually reap lots of money, but it takes time. We had upfront costs, making sales deals took a lot of time and we burned money without income.
The better model would have been: Do consulting on knowledge management and start with an open source product.
We did consulting to companies on how to do knowledge managment, use wikis etc. But we didn’t take any money for it, because it was part of our sales process. Focusing on cosulting and billing people would have created a steady income.
I did get into open source later with SnipSnap. SnipSnap took (a small part of) the ideas from the startup (wiki and blog) and was relased as an open source tool. Lots of people downloaded the software and installed it on their desktops. We really made installing SnipSnap easy, so it spread fast. I’ve talked to a boss of a very big software and consulting shop and he told me, wikis would never work for them: too chaotic and not structured enough. Well - in fact I knew that there were several SnipSnap installations in his company :-) As others are practicing now we could have gotten our foot in the door with an open source project, then sell support and enterprise features on top. Companies later paid us money to put features into SnipSnap, make it scale better and for other enterprise thingies. But in 1999 we didn’t know as much about software business models as we know now.
What can you learn from my mistakes? Not sure, but start selling. I’ve learned a lot though about software managment, products, business models, money and being a CTO.
Thanks for listening.
Looks good on paper, my first try results in
Crash dump was written to: erl_crash.dump
init terminating in do_boot()
without any understandable error message. Should try CouchDB.
PS: Rebuilding everything on Debian from source. We’ll see
PPS: Didn’t help :-(
PPPS: CouchDB won’t compile, it’s not finding an Erlang kernel header file. This is why I love Java and hate C. I forgot how painful compiling and installing builds with make and .h-files was.
PPPPS: Found a version which did compile. Java API doesn’t work.
PPPPPS: Back to MySQL which works. There seems a lot work needed to dethrone MySQL
PPPPPPS: Tried the same with Ubuntu 8 without success
In my last post about “50k lines of code considered large?” I’ve wondered about large code bases and the different perceptions on what a large code base is. I came to the topic because of a blog post: “The Maintenance myth” by Ola Bini. One minor point he makes about maintanence is lines of code in dynamic languages. I know maintanence is mainly about technical debt. But I’m interested in how lines of code factor into maintanence problems. Ola says
“(very large code bases is 50K-100K lines of code in these languages)”
pointing to Ruby and Python. In a reply to a comment from me Ola writes “I would consider 50k-100k in Ruby to be very large, yes, definitely. I know of Python code bases between 100k and 200k, but that’s about the largest I’ve heard of.” With my Java background - and some Ruby and Python background mainly from the 90s - I consider very large applications to be much bigger, perhaps 500k to 1M for very large - not 50k lines as for example SnipSnap has ;-) The Linux kernel contains between 6.4M and 10M lines of code depending on the way you count. There seems to be a huge difference in what people consider very large. There could be several reasons:
Considering the second hypothesis the factor should be between 10x (50k compared to 500k) and 20x (50k to 1M) for things people consider very large - taking Ola and his coworkes and me (I didn’t ask my team ;-) as a very small sample set.
Therefor I’ve expressed an example in code. The example is an application fragment for managing songs - the idea coming from the common Ruby introduction. I’ve chosen to compare Python and Java because Python is considered by some people a more mature language and used by larger projects than Ruby and because I did more and bigger projects in Python (my Ruby experience is only some years of coding web applications in Rails and writing an OR mapper and web component framework in Ruby by myself: “Convention over Configuration Framework in Ruby from 2002″). Someone could do a Ruby comparison :-)
People may be surprised, but Java development and style - at least avantgarde - has changed over the last 13 years. So the example might not look like you think that Java should look. It reflects the style I would right now write green field Java code. It is inspired by Domain Driven Design and functional principles (for more about DDD and composite oriented programming in Java see Qi4J and real world Qi4j). In true DDD style I would prefer more objects like Name and Duration - see “Never, never, never use String in Java (or at least less often :-)” - but I’ve cut the example for brevity. Some people would not use a SongList domain object but a List directly. From my point of view, if SongList is a Domain Object and not an implementation detail, you should create a class and not use a List. So I’ve used a list.
A note on formatting: Lately parts of the wave front surfing Java developers switched to one line formatting of small methods, something which helps code readability and understanding a lot (you’ll see). IntelliJ IDEA does support this as a formatting option. It’s a very good feature in IDEA but to my shame I only detected it very late, but glad I did as it’s so much better this way.
For manipulating, filtering and transforming lists I currently use Google collections. For an introduction see here. Google collections make working with lists much easier.
The REST part of the application is missing in Python as I have not enough knowledge to write the code with a state of the art REST framework. Perhaps someone could fill me in. In the Java example I’ve chosen to create the JSON and XML on my own without an automatic mapper. Automatic mappers do exist though, one could use JAXB. The code for the builder is explained in a previous post.
One cautionary note: The only Python books I have are “Internet Programming With Python” and “Programming Python”, the first edition, both from 1996. Sorry that my Python is rusty, all correcting comments or comments on how to do it better are welcome. Please focus on better, not on shorter.
On to the examples:
Java
public class Song extends Entity<SongId, Song> {
public Property<String> name;
public Property<Integer> duration;
public One<Artist> artist;
public Song(String name, int duration, Artist artist ) {
this.name = read(name);
this.duration = read(duration);
this.artist = read(artist);
}
}
public class Artist extends Entity<ArtistId, Artist> {
public Property<String> name;
public Artist(String name) {
this.name = read(name);
}
public String toString() { return name.get() }
public Artist artist(String name) { return new Artist(name); }
}
public class SongList implements Iterable<Song≶{
private List<Song> songs = newArrayList()
public SongList addSong(Song song) { songs.add(song); return this}
public Iterator<Song> iterator() { return songs(); }
public Iterator<Song> filter(Predicate<Song> p) { return filter(iterator(), p); }
}
Some example usage:
// Not counted, as usually the pattern is
// SongList list = ... from Database ... or
// SongList list = ... from UI ...
SongList list = new SongList()
.add(new Song("S1", 5, artist("A1"))
.add(new Song("S2", 8, artist("A2"))
.add(new Song("S3", 13, artist("A3"))
// Print all songs
for (Song song: list) {
System.out.println( song.name() + " by " + song.artist() + "(" + song.duration() ")" );
}
// Print all songs with duration smaller than 10 (minutes)
final Predicate<Song> durationLowerThan10 = new Predicate<Song>() {
public boolean apply(Song song) { return song.duration.get() < 10; }
}
for (Song song: list.filter(durationLowerThan10) {
System.out.println( song.name() + " by " + song.artist() + "(" + song.duration() + ")" );
}
and a simple REST service which returns a JSON or XML (depending on the request) representation of the song list.
// Example REST Service
// Returning JSON and XML to a REST call, without automatic mappers like JAXB
public class SongListResource {
@Inject ListService service;
@GET @Path("/songs/{listId}")
@Produces("text/xml", "application/json")
public Node getList(@PathParam("listId") String listId) {
SongList list = service.listForId( listId );
return $("songs", new List>Song<(list) {
protected Node item(Song song) { return $("name", song.name() );}
});
}
}
The example in Python
class Song:
def __init__(self, name, duration, artist):
self.name = name
self.duration = duration
self.artist = artist
class Artist:
def __init__(self, name):
self.name = name
def __str__(self):
return self.name
class SongList
def __init__(self):
self.songs = []
def add(self, song):
self.append(song)
some example usage
# Not counted, as usually the pattern is
# songList = ... from Database ... or
# songList = ... from UI ...
# Not using SongList, the examples should be the same though
# or not?
songList = [ Song("S1", 5, Artist("A1")),
Song("S2", 8, Artist("A2")),
Song("S3", 13, Artist("A3")) ]
for song in songList:
print "%s by %s (%d)" % (song.name, song.artist, song.duration)
# could provide print method to list
for song in (song for song in songList if song.duration < 10):
print "%s by %s (%d)" % (song.name, song.artist, song.duration)
The example is very short and perhaps not very meaningful. One would need to do more empiric research (e.g. comparing FP to LOC in different languages). And perhaps some readers will provide addtional information. So the conclusion is preliminary and will be updated. Counting the lines of code there are 33 NCSS in Java and 19 NCSS in Python. Java has around 1.7 times the LOC of Python from my example. Taking the hypothesis above this could mean several things:
I can’t comment on the first conclusion. The second conclusion means, someone would need to compare two framework examples, say the song list in Seam and Django. The third conclusion is very interesting. It would mean that people consider applications written in Python very large although they (relatively) contain a lot less lines of code. Ola considers 50k to 100k very large, with a factor of 2x this would make 100k to 200k of Java lines. I can’t speak for most Java enterprise/startup developers, but as I consider 500k to 1M very large, Ola and I differ by a factor of 5x of what very large is. I only can speculate what’s the reason for this.
The first conclusion somehow fits with another quote from Olas post: “And it’s interesting, the number one question everyone from the static “camp” has, the one thing that worries them the most is maintenance.”. They may have seen “very large” applications contrary to the “dynamic camp”.
“Of course, this is totally anecdotal, and maybe these guys are above your average developer.”
I’m glad to provide a step (small one) from the anecdotal to the empiric and from the empiric of this post I don’t think people considering 100k of lines “very large” are “above your average developer”.
Another side note: “But in that case, shouldn’t we hear these rumblings from all those Java developers who switched to Ruby? I haven’t heard anyone say they wish they had static typing in Ruby.” Perhaps because they do green field (not brown field) development? And you need to develop for several years in one application to make it a brown field? And it takes several years to accumulate enough technical dept? Because most of them just started and don’t do “very large” applications?
Other interesting stuff:
Lots of open questions and I would be very interested in other opinions and other examples - and to explore the topic further.
Thanks for listening to this very long post.
Update: Ryan (see comments) supplied a version of a function in C and Python and after removing the hand memory allocation code and the Python interface code of the C version, the factor is 2.2x (38 to 17 NCSS). Thanks.
Update 2: Looking at Oloh (see comments) the factor of Java and Python is 4x. Very large base of examples. One would need to check the types of programs.
Update 3: An old article I’ve found again “7 reasons I switched back to PHP after 2 years on Rails”. An interesting info: After going to Rails and coming back, with the Rails knowledge the PHP app was reduced in size “- … and much more. In only 12,000 lines of code, including HTML templates. (Down from 90,000, before.)”. Looks like rewrites or prior experience in the domain reduces code size. Could explain Olas experience with Java developers who switched to Ruby. Came to this article again through a comment by Harry Pynn “Point number 7 is that programming languages are like girlfriends: The new on is better because you are better. Could it be that people moving to dynamic languages from static languages find it easier to write maintainable code having honed their skills with a static language?” on Frank Carvers blog.
According to this blog post:
“In the recent event Google Developers Day, Bangalore, the Keynote speaker Prasad Ram said that Google Appengine will now support Java. [...] “Java”, said the other speaker, “was chosen based on the community feedback.”
Splendid! I hope this includes Scala somehow. This would at last be an easy way to deploy Java web applications, one thing up to now only PHP could claim (and to a lesser amount Ruby and Python).
Ola Bini considers 50K of lines as very large:
“I know several people who are responsible for quite large code bases written in Ruby and Python (very large code bases is 50K-100K lines of code in these languages).”
This explains a lot.
And the blog post made me think. We’ve written >50K code bases in Python in the 90s in a small development shop (<10 developers). I don't consider this “very large”. Large or very large starts for me at sizes when one developer cannot possibly know all the code (independently of the language) or cannot have a good overview.
As I see now that Ola Binis blog scrambled my comment, I repeat it here for reference
“The Maintenance myth”
[snip snip snip]
“Has there been any research done in this area?”
Nice blog post, if you cut out the middle. Interesting calling something a myth and then asking about research in the end.
“(very large code bases is 50K-100K lines of code in these languages).”
100K is very large? I wrote some projects in a two person team and reached 50K of lines. This is rather small. We did 50K Python programs in the 90s in a small development shop (<10 developers). Very large starts (for me) at 1M LOC.
(I don't like LOC as a metric though, FP or "Thought points" are much better because they are more comparable between languages and make more sense: A developer has to think about every "thought point" => more thought points = more complexity & more effort).
@Seo: “Codes written in dynamic languages tend to be shorter than codes written in static languages doing the same thing, and I think code size is the most important factor in maintenance.”
I don’t think Scala is much larger in LOC than Ruby.
And though Lisp & Haskell may have less LOC, they have a lot of Thought points because they have a high density of thought points whereas Java has a very low density with lots of noise in between.
Thanks for listening.
Update: Concerning my comment to Seo
An Ruby example
class Song
def initialize(name, artist, duration)
@name = name
@artist = artist
@duration = duration
end
def how_long
"{@duration} minutes"
end
end
or with idiomatic Scala
class Song(val name:String, val artist:String, val duration:Int) {
def howLong = duration + " minutes"
}
or more similar to Ruby:
class Song(aName:String, aArtist:String, aDuration:Int) {
val name = aName
val artist = aArtist
val duration = aDuration
def howLong = duration + " minutes"
}
Another Update: Marcos suggested
class Song
def initialize(name, artist, duration)
@name, @artist, @duration = name, artist, duration
end
...
as more idiomatic Ruby. Thanks.
David Pollak was right about XML and JSON, but perhaps in a different way. XML cannot be converted to (clean) JSON.
Suppose we have a shopping cart in XML which we want to convert to JSON:
<cart>
<items>
<item><name>one</name></item>
<item><name>two</name></item>
</items>
</cart>
One representation in JSON would be (cart could be omitted):
{ cart: { "items": [ { "name": "one" }, {"name": "two"} ] } }
We convert a list of nodes with the same name to a JSON array, xml2json-xslt does this for example. What happens if we only have one item in our shopping cart?
<cart>
<items>
<item><name>one</name></item>
</items>
</cart>
Then our converter cannot detect that items is a list and will convert the XML to:
{ cart: { "items": { "name": "one" } } }
which is semantically something completely different. And very unpleasent for the receiver of our JSON code, because sometimes he gets an array and sometimes an object.
One way to solve the problem is to annotate the XML (looks ugly but works):
<cart>
<items type="list">
<item><name>one</name></item>
</items>
</cart>
and adding an additional condition to the XSLT
or ../@type[.='list']]
or namespacing (doesn’t work yet, we get lots of namespaces and misuse XML namespaces) ?
<cart>
<list:items>
<item><name>one</name></item>
</list:items>
</cart>
So David was right, I’m not sure he new why ;-)
Thanks for listening.
Tim Dysinger wrote some time ago about his unpleasant experiences in a Scala chat channel. My experiences with the Scala community has been nicer. There sometimes is a little bit of FP zealotry but mostly people are willing to help. Perhaps it was his attitude towards the language with “the syntax is verbose, ugly and robotic.” when entering the chat which lead to his unpleasant welcome. Perhaps it was something else. The comments are closed on his post, so people couldn’t reply and the context is lost. I often write provoking things, at least I keep the comments open and get my share of ugly replies.
Being arrogant and insulting: “Notice that comments aren’t enabled, Scala nerds. Go comment somewhere else.”
I’m so very happy to not be part of the Ruby community. It was nice in 1998 and went downhill from there.
Update: Thanks to the comment, you do not need to google: Academics and Ergomaniacs
Update 2: No need to read the IRC log
(10:09:30 AM) DRMacIver: So, just for the record, I looked up the source code for Object.equals
(10:09:49 AM) DRMacIver: public boolean equals(Object obj) {
(10:09:49 AM) DRMacIver: return (this == obj);
[...]
(10:11:54 AM) dysinger: fuck you guys
[...]
(10:13:11 AM) dysinger: fuck you drmaciver - you and I have debated endlessly on other topics.
[...]
(10:14:37 AM) dysinger: eat a dick egomanics
[...]
(10:14:58 AM) dysinger left the room (Kicked by DRMacIver (I've already told you you're not my type.)).
To be honest I would have kicked him out after the first “fuck”.
And from the comments “dysinger said… So in the end it’s not about the code. We are all smart people [...]“.
Sorry. Nope.
“I shouldn’t have swore. That was bad on my part. [...] David MacIver can still bite my ass. [...] WTF?!”
Talk about idiocy.
After reading the IRC log: I’m so so very happy to not be part of the Ruby community.
PS: I’m sorry to use the word ghetto.
Hmm[*]. Not sure if this is true (with CDATA, #Text and @attributes handled in some converters). For me the problem is more that there are too many ways to convert XML to JSON. For exampe the Badgerfish convention. Or the the Google and Yahoo versions. Or the XML.com way. And the Parker convention.
But the ways in Javascript to convert XML to JSON are either slow, very basic, use XSLT, use nasty Regex or cannot create simple JSON which feels JSON like.
* Note to self: Should start using Twitter for this.
Update: Any ideas for a good XML to JSON conversion which feels JSON like (no need to be bidirectional)?
Update 2: I currently use XSLT with nice results, Safari doesn’t work yet and neither does Chrome. More to come.
Together with one of the senior developers on my team I gave a speech at the the first Berlin Java conference called Berlin.JAR. The topic was about how to develop applications for the web without a (traditional serverside) web framework. There is a wave towards rich AJAX applications with GUI logic in Javascript. Two forays are SOFEA and SOUI. The speech covered examples of how to design web applications without a web framework with rendering in Javascript and a client side message bus, using REST, Jersey, OpenAJAX, PURE JS and jQuery. My slides in an English version can be found on SlideShare or here.
The really nice thing is that backends can be written - independently - with Ruby, Python, Javascript, Java, Clojure, Scala, Erlang, OCaml or any other language. As integration doesn’t happen on the server but in the browser (late and lazy integration), it’s faster (asynchronous) and easier than on a server.
Thanks for looking.
Scala has native support for xml in the language via scala.xml.Node
val message = <message>Hello world</message>
There is an excellent book called scala.xml on XML and Scala so this post won’t go into more detail.
Using the XML support in Scala makes it easy to write REST applications with Jersey. I’ve shown how to create JSON with a JsonBuilder in Scala before.
We need a Resource to handle our REST requests. As a response to a GET request on /helloWorld/xml we create a Scala XML node:
@Path("/helloWorld")
class HelloWorld {
@Path("/xml")
@GET
@Produces(Array("text/xml"))
def helloWorld() = {
Hello World
}
}
Jersey needs to know how to translate an object of scala.xml.Node to a HTTP response. This is usually done by implementing a MessageBodyWriter that maps an object and a mime type - scala.xml.Node and text/xml in this case - to a response.
@Provider
@Produces(Array("text/xml"))
class ScalaNodeAdapter extends MessageBodyWriter[scala.xml.Node] {
def isWriteable(dataType:java.lang.Class[_],
typ:Type,
annotations:Array[Annotation]) = {
classOf[scala.xml.Node].isAssignableFrom(dataType);
}
def writeTo(node:scala.xml.Node, writer:Writer) {
writer.write(node.toString)
}
def getSize(node:scala.xml.Node) = -1L
def writeTo(node:scala.xml.Node, aClass:java.lang.Class[_],
typ:Type,
annotations:Array[Annotation],
mediaType:MediaType,
stringObjectMultivaluedMap:MultivaluedMap[String,Object],
outputStream:OutputStream) {
val writer = new OutputStreamWriter(outputStream);
writeTo(node, writer)
writer.close()
}
}
Voila, we now get <message>Hello world</message> when calling /helloWorld/xml.
Thanks for listening.
There is a lot of holladi about DSLs (in Ruby and Scala and everywhere). The secret problem with DSLs that nobody talks about is easy to explain: Growing and designing a language has been shown to be hard. Most people who think they could solve a problem with a DSL are not good language designers => disaster.
(Following the blogosphere there are lots of proposed DSLs which are ugly, inconsistent, hard to read and not useful)
When looking at server numbers from Facebook, Youtube or Linkedin, it would be nice to have a metric to compare concurrent users/server over several architectures (becnhmarking). Is there something like that?
There are some discussions about Scala vs. Clojure - which one could replace Java on the VM.
I think the object oriented features of Scala make the language more usable for real world applications.
But the idea of Clojure - tight integration with Java through Iterable and Iterator, implementing Java interfaces, but keeping immutable structures, compared to Scala which creates it’s own incompatible versions, should prove much more successful. I like that definitely way better, Scala should adopt that approach. And of course implementing STM in Clojure is genius - lots of people talk about STM and it could be the next big thing for sharing state in distributed applications.
Thanks for listening.
Update: Sequences in Clojure http://blip.tv/file/734409