I really like Hibernate, but I also don’t know a tool that would be nearly as powerful and deceptive at the same time. I could write a book on surprises in production and cargo cult programming related to Hibernate alone. It’s more of an issue with the users than with the tool, but let’s not get too ranty.
So, here’s a recent example.
Problem
We need a background job that lists all files in a directory and inserts an entry for each of them to a table.
Naive Solution
The job used to be written in Bash and there is some direct SQL reading from the table. So, blinders on and let’s write some direct SQL!
for (String fileName : folder.list()) {
SQLQuery sql = session.getDelegate().createSQLQuery(
"insert into dir_contents values (?)");
sql.setString(0, fileName);
sql.executeUpdate();
}
Does it work? Sure it does.
Now, what happens if there are 10,000 files in the folder? What if you also have a not so elegant domain model, with way too many entity classes, thousands of instances and two levels of cache all in one context?
All of a sudden this trivial job takes 10 minutes to execute, all that time keeping 2 or 3 CPUs busy at 100%.
What, for just a bunch of inserts?
Easy Fix
The problem is that it’s Hibernate. It’s not just a dumb JDBC wrapper, but it has a lot more going on. It’s trying to keep caches and session state up to date. If you run a bare SQL update, it has no idea what table(s) you are updating, what it depends on and how it affects everything, so just in case it pretty much flushes everything.
If you do this 10,000 times in such a crowded environment, it adds up.
Here’s one way to fix it – rather than running 10,000 updates with flushes, execute everything in one block and flush once.
session.doWork(new Work() {
public void execute(Connection connection) throws SQLException {
PreparedStatement ps = connection
.prepareStatement("insert into dir_contents values (?)");
for (String fileName : folder.list()) {
ps.setString(1, fileName);
ps.executeUpdate();
}
}
});
Other Solutions
Surprise, surprise:
- Do use Hibernate. Create a real entity to represent
DirContentsand just use it like everything else. Then Hibernate knows what caches to flush when, how to batch updates and so on. - Don’t use Hibernate. Use plain old JDBC, MyBatis, or whatever else suits your stack or is there already.
Takeaway
Native SQL has its place, even if this example is not the best use case. Anyway, the point is: If you are using native SQL with Hibernate, mind the session state and caches.