Saturday, 6 August 2011

Unlimited Detail Technology

Without a doubt, the technology story of the week is Euclideon's Unlimited Detail demo: an apparently revolutionary approach to graphics rendering that sees the end of polygons, replaced by billions upon billions of "atoms" that allow for an infinite level of detail in any given game scene. In short, Euclideon appears to have come up with a new approach to voxel rendering that seemingly has much potential.

At the time of writing, the demo has been viewed almost 1.8 million times on YouTube, and has engendered fierce debate on the internet with the harshest criticism coming from Minecraft developer Notch, calling out Euclideon's engine as "a scam".

"It's a very pretty and very impressive piece of technology, but they're carefully avoiding to mention any of the drawbacks, and they're pretending like what they're doing is something new and impressive. In reality, it's been done several times before," Notch observes.

"They're hyping this as something new and revolutionary because they want funding. It's a scam. Don't get excited. Or, more correctly, get excited about voxels, but not about the snake oil salesmen."

Other developers, including id software's John Carmack, have been more measured in their response, mostly because a variation of voxel technology - the Sparse Voxel Octree (SVO) - is one of the most promising approaches being considered for next-gen rendering, where the raw horsepower and RAM could conceivably be available to make the approach pay off. However, Carmack rules out a current-gen offering based on the Euclideon tech, something the Unlimited Detail company takes issue with.

As it is, the technology itself isn't as new as you might think it is. Developer Bruce Robert Dell was known to have shown video demos to software developers back in 2008, and over the last few years has built up a small company and claims to have attracted significant investment. According to people who've seen Dell's 2008 presentation, his work actually stretches back several years before that.

"Last year we were just two people and I suppose we're what some people would call a garage job," Dell tells us.

"Since that time we've started a company, we have nine employees, we've received investors and got one of the largest grants in the country from the government. Our board of directors are some of the top people in Australian software, and our chairman of the board is the former CEO of one of Australia's largest technology companies. Having a proper company with employees has made a big difference."

While the scope of the operation has increased significantly and the demos have obviously improved over the years, it seems to be very difficult to get any kind of information from Euclideon about how Unlimited Detail works or even what the underlying principles are. Bruce says that he doesn't want to give his secrets away, however, he appears to have been more open about the technology back in 2008:

"The system isn't ray-tracing at all or anything like ray-tracing. Ray-tracing uses up lots of nasty multiplication and divide operators and so isn't very fast or friendly," Dell posted on the Beyond3D forum.

"Unlimited Detail is a sorting algorithm that retrieves only the 3D atoms (I won't say voxels any more it seems that word doesn't have the prestige in the games industry that it enjoys in medicine and the sciences) that are needed, exactly one for each pixel on the screen, it displays them using a very different procedure from individual 3D to 2D conversion, instead we use a mass 3D to 2D conversion that shares the common elements of the 2D positions of all the dots combined.

"And so we get lots of geometry and lots of speed, speed isn't fantastic yet compared to hardware, but it's very good for a software application that's not written for dual core. We get about 24-30FPS [at] 1024x768 for that demo of the pyramids of monsters."

As for the brand new Island video, Dell is open about the system specs required to run it and the performance he gets:

"The latest demo was running on our office laptop which is 2GHz Core i7. It ran at 20FPS [at] 1280x720 purely in software without touching the 3D part of the GPU, but we certainly haven't added all our optimisation yet. I think next time you will be pleasantly surprised," he reveals.

Beyond that, getting much in the way of information about how Unlimited Detail actually works proves fruitless, with Dell often choosing to be humorously defensive in answering even basic technological questions, or simply evading them altogether. When asked how the tech differs from approaches like Sparse Voxel Octree and GigaVoxels, Dell simply points out that the run-time speeds of these techniques are much slower than his, but doesn't offer more detail. Asked to clarify how much memory an atom uses, he suggests that an accurate answer would simply be unbelievable to us, and rather than try to win us over and evangelise his tech, he pursues a "wait and see" line of argument.

"If we were making our world out of little tiny atoms and had to store x, y, z, colour etc for each atom, then yes it would certainly use up a lot of memory," he acknowledges.

"But instead we've found another way of doing it. I could say we use less memory than what the current polygon system uses, but if I did that I think I'd exceeded my quote of unbelievable claims for the day. So we'll leave that for future demonstrations."

We also probed on how compatible Unlimited Detail is with the traditional vertices/skeletal animation/surface texturing approach used in current video game production. Bruce Dell readily admits that the lack of animation in his demos is the biggest point of criticism.

"Yes we can do animation, but it's not finished yet," he says.

"Last time, we learnt that if we were to put anything on the internet that wasn't finished, there would be hordes of forum people who are more grumpy than that donkey from Winnie the Pooh who would point the finger and say 'look at that, that doesn't look as good as polygons' no matter how hard we tried to say 'but we're only half done'. I'm sure our supporters understand the wisdom of us being silent on the topic of animation until it's completed."

Another criticism levelled at the demo is that the same objects are repeated across the island, the inference being that replication is being used to disguise basic limitations of the technology. Dell attempts to put the new video into context, saying that the decision to go ahead with a new demo only came a few weeks ago.

"Our aim was to show the technology, not necessarily beautiful graphics, I think we succeeded in our task, it's not a limitation of the technology, it simply came down to not having enough time to make more objects," he explains.

"We only have one artist and the poor guy has been slaving away to the point that even Cinderella would have pity on him - please don't accuse him of too much laziness. As said before, we're a technology company not a games company - that is all the art that could be included in the demo in such a short amount of time."

A Euclideon supplied comparison of ground detail using a conventional polygon renderer compared with the same effect seen in Unlimited Detail.
Polygon Ground

Unlimited Detail Ground

Fair enough perhaps, but what is curious about the demo is that the orientation of the objects is identical: they all face the same way. A very basic form of variety that could have transformed the look of the demo radically would have been to rotate the foliage objects by varying degrees.

The very simplistic lighting and materials also stand out, but again, Bruce Dell suggests that this is an early demo and we should not be judging it so harshly in its current state.

"When it comes to lighting, as we said in the video, it's not quite finished yet. Unlimited detail is a geometry system, like polygons are a geometry system. Lighting is something separate," he tells us.

"We have working examples where Unlimited Detail is using the lighting from the graphics card, they are entirely compatible. However, we're also working on a few lighting techniques of our own which you will see in the future."

Another Euclideon comarison, this time covering off the incidental details like rock - low poly models in conventional engines, but transformed with Unlimited Detail.
Polygon Rocks

Unlimited Detail Rocks

To his credit, Dell does acknowledge the importance of current production workflows involving industry standard tools such as 3DS Max and Maya. They're all polygon-based, so Euclideon has developed a tool that converts these polygon-based constructions into point cloud data that can slip into the Unlimited Detail engine.

"Regarding polygon conversion, originally we were seen as the enemy of polygons: we constructed shapes out of little atoms, they were constructing shapes out of flat panels," Dell says.

"The games developers we were in contact with didn't want their development pipeline to be adjusted radically. This makes sense when you consider all the current artists, their skills and technique, and the fact that the current tools in 3DS Max, Maya and others are very, very good.

"So instead, we decided the best way forward would be to build a polygon converter. In effect what you have is a system that is converting polygons to little atoms, and then running those little atoms with our Unlimited Detail technology. But to the artist, they feel like they're just using unlimited polygons, their pipeline hasn't been changed in any huge way."

But again, how does it work? What we actually asked him was whether they are simply storing an atom for each point in the texture along each polygon surface, or if there is a more sophisticated technique at work: a pretty fundamental question any developer using these tools would ask - but even a basic question like this gets no straight reply.

"Well, I don't want to speak too much on our technique at this point in time," was the answer we got to the specific point we actually asked.

And this is the main issue we have with Unlimited Detail at the moment. There are known problems with the implementation of voxel technologies and most of the issues are present in the Euclideon demo. The company claims it's doing something new and exciting, and while the demo is acknowledged - even by Notch - as impressive in many ways, the applications within the gaming arena have yet to be proven. Bruce Dell's responses to questions that would clear up these issues are by turns evasive, somewhat defensive and either intentionally or unintentionally comedic.

Some of the discussion being generated by this in the development community is intriguing though. Not everyone is going straight for the jugular on the merits of the demo, but again, the same questions are being raised that we attempted - and failed - to get answers for.

"The geometry quality is excellent in these videos. With professional artists and better lighting and image post processing systems they could easily exceed the current game graphics quality in small static environments," observes Trials HD's Sebastian Aaltonen.

"However since they do not mention anything about their 'revolutionary' data compression algorithm, I am quite sceptical how this technology could be used in a real game. A single high quality point cloud rock can be over 100 megabytes of data. Polygons combined with displacement maps (tessellation for low frequency detail, and pixel displacement techniques for high frequency detail) offer very high compression ratio compared to high polygon meshes (and point/voxel data). Displacement maps are also very easy to stream in and out of memory (virtual texturing or any other existing texture streaming technology can be used)."

In line with John Carmack's comments, it all seems to point to voxel-based technology of some description only finding a home on next-gen consoles, but Bruce Dell takes issue with that.

"Firstly I'd like to say that I greatly respect John Carmack. In light of the fact that we haven't released real-time demos, his statement is sensible, sane, reasonable, but incorrect," he says.

"We have too much respect for Mr Carmack's contributions to the 3D graphics industry to speak negatively of him and understand his comments in the light of the graphics systems he is aware of."

Bruce Dell recognises that tessellation can add 'unlimited detail' to a polygon renderer, but reckons that 'tessellated bumps' don't measure up to his Unlimited Detail, asking us to consider this comparison. It is worth pointing out that Dell took the tessellation image from a DirectX sample that's very simplistic and doesn't compare with production examples in actual games.
Tessellated Height Map Rocks

Unlimited Detail Rocks

In putting this article together, we were hoping that Euclideon and Bruce Dell might like to take the opportunity to promote the technology, especially as game developers are a component of our audience. To that end, we consulted technical personnel from the games industry who've worked on many high-profile multi-platform games and framed our questions based on their input. While Dell himself comes across as a friendly, humorous guy, the lack of any real technical detail at all in his responses was disappointing.

We put it to him that we were posing the questions potential customers for his technology would be asking and suggested that his answers would only polarise opinion on Unlimited Detail still further, and invited him to expand on his responses.

"I know this might sound strange but at this time we aren't actually trying to get great attention or credibility until our product is finished," Dell says.

"Our intention was to put a little video on YouTube to tell our fans and supporters what we had been doing this year - we never expected it to get 1.5 million views in three days. The negative scam sentiment surprised us at first, but then we realised it probably helps... When we are finished and release real-time demos perhaps there are some points for us in the fact that so many people were wrong about us."

Dell reckons that his technology is being pigeon-holed in the context of existing paradigms, and he believes it's something completely new that industry experts simply do not understand. We went into this expecting a technical discussion but for whatever reason, Bruce doesn't really want to talk about Unlimited Detail on those terms.

"I think the tech community are asking [if it's] a voxel system, a splat system, or ray-tracing and are trying to get enough info to box us as something known. The fact that it's not any of those systems and doesn't resemble any of those systems isn't the answer people are looking for and I am not about to try and hand over all our secrets at this point in the project," he says.

We asked Bruce Dell what stands in the way of getting the technology into the hands of game-makers and what's left to complete.

"I'd like to answer that, but I know that anything I say, regarding what we're still working on, there will be people who point and say 'Ha! They are still working on it, it's not finished! It will never be finished!' and do the dance of joy rejoicing in our interpreted destruction."

Makes me wonder if the clever people at OnLive, Mova and Rearden, could do something with Euclideon's Unlimited Detail technology. Since OnLive uses powerful data centers for cloud gaming, this Unlimited Detail technology could be a great fit for them and with the help of DIDO, OnLive could enable the use of Unlimited Detail on mobile devices.

SOURCE: Eurogamer.

1 comment:

  1. Dell reckons that hisD3 items technology is being pigeon-holed in the context of existing paradigms, and he believes it's something completely new that industry experts simply GW2 Cd keydo not understand.