Size limit for Nature Terrain Generator?


https://forum.reallusion.com/Topic487974.aspx
Print Topic | Close Window

By TimothyMasters - 4 Years Ago
The more I use Nature Terrain Generator, the more I realize that it's an incredible, wonderful tool that was not quite ready for release.  I'll put aside for the the issue that when morphing tiles it is ridiculously easy for the tiled surface to suffer terrible tears where they should have remained connected.  It's annoying having to scan every seam from every angle just to make sure there are no tears, and then do it again after fixing the tears to make sure the fix did not open a new tear.
Anyhow, my real issue involves generating large terrains.  I need a 20x20 surface, which is just 80 meters on a side, not excessively large for an area for character exploration.  My generation ran for four hours before I gave up and killed it.  I do wish there were a progress bar!  So I figured I'd just do four 10x10 surfaces and stitch them together.  I was happy when my first 10x10 took just 8 minutes to generate.  I slid it 4000 units left to get it out of the way and then tried to generate a second 10x10.  An hour later it was still humming away, and when I clicked on the screen to see if it was still alive (Windows called it unresponsive), iClone just vanished, a total crash.

So, is there an upper limit?  Is it illegal to generate a second terrain when there is already one in the project?  Is this just yet another NTG bug?  Do you suppose the developer has any plans for fixing this highly promising program?  This is so frustrating!
By Kelleytoons - 4 Years Ago
I'm no expert on it but first things first.

First of all, make sure there are NO other objects in your scene.  iClone has a terrible problem when objects in a scene get over a few dozen, and in particular it will manifest itself when running Python programs.  So you should be starting from a completely brand new project to generate your terrain - you can always add other things later after you merge the surfaces.

Now - as to that.  After you generate your first piece of ground, make sure you do merge all the pieces.  IOW, go into your scene manager, select them all, modify/detach, then merge them all together as one accessory.  Okay, now you should only have the one object in your scene (this won't allow you to modify it anymore, so make sure you get it looking how you like it first.  Now generate the next piece.

If you have problems even with that, save that one piece out as an accessory, delete it and start with a blank scene again, and start again.  Now, if you need to modify the various elements you may have to be create - let's say you want the edges of each new piece to be able to be modded so you can join them properly.  Okay, then just merge the interior pieces and save them as one piece, and then save and merge each edge as you generate and shape it.  

Hopefully that all makes sense - if not, I can make a video showing the process.  Again, it's a PITA, but I suspect it's (partly? mostly?) due to this being a Python script AND iClone not dealing well with dozens of objects.  But you can work around those limitations.
By TimothyMasters - 4 Years Ago
Thanks for all that advice.  I guess you are saying it's more likely a problem with iClone rather than NTG?

I've been using your third suggestion to get around the problem.  I just generate a quarter of my terrain, tweak it to perfection, merge, save, delete, do the next quarter, etc.
The problem is that I really wanted to see my ENTIRE terrain when I make the topography.  Now I have to visualize the other parts as I generate a part.  It's a nuisance, but I guess I have to live with it, perhaps for a very long time.
By TimothyMasters - 4 Years Ago
Thanks for all that advice.  I guess you are saying it's more likely a problem with iClone rather than NTG?

I've been using your third suggestion to get around the problem.  I just generate a quarter of my terrain, tweak it to perfection, merge, save, delete, do the next quarter, etc.
The problem is that I really wanted to see my ENTIRE terrain when I make the topography.  Now I have to visualize the other parts as I generate a part.  It's a nuisance, but I guess I have to live with it, perhaps for a very long time.
By pinguintje - 4 Years Ago
I did some work with the Nature Terrain Generator.
The best size for me was 100 planes. More then 100 planes kills the framerate.
For me the quickest way to start is:

-Create a new group 5x5, subdivision 64x64, 4K (it takes about 10 seconds)

https://forum.reallusion.com/uploads/images/a5bf5815-37ab-4b1a-8bee-ed9b.jpg

- Select the 25 planes and use Ctrl+move  and snap them with Ctrl+M

https://forum.reallusion.com/uploads/images/6793b1e7-61f5-45f1-b0aa-61e0.jpg

- Do this 2x again. (choose your own direction)

https://forum.reallusion.com/uploads/images/6e41f129-fe07-400b-a573-94bf.jpg

-Now you have 100 planes. It takes me less then 5 minutes and iClone can handle this without problems.

Working with 100 planes means you have to scale your plants, trees characters etc. to 30-50%

Keep in mind Kelleytoons' good tips.

It's now time for you to create hills, water etc.:):):)

Best Regards,
Arie


By TimothyMasters - 4 Years Ago
Kellytoons - Well, my 'toy problem' approach worked, but as soon as I scaled it up to the size I needed, iClone crashes.  I have to admit ignorance on one thing you said:
"go into your scene manager, select them all, modify/detach, then merge them all together as one accessory."
I'm afraid I've never done that, and don't have a clue how to do it.  When I select them all and go to the Modify menu, all I see is 'Convert to terrain' which quickly crashes iClone, and some other things that seem inappropriate.  Can you please be a little more specific, or direct me to a topic in the manual.  Thanks!
By TimothyMasters - 4 Years Ago
Arie - Thanks for that detailed response!

I did what you did and got exactly your result.  But now I have a mystery.
I selected all tiles and converted them to terrain, and it went perfectly.
Here's the mystery, and it's absolutely repeatable, not a one-time fluke
If I create a 9x9 in one shot I have 81 tiles, fewer than your 100.  But when I select them all and convert to terrain, iClone goes nuts (empty spaces, some in the scene list having physics, some not, etc.) and then it does a hard crash.

So... Why does getting 100 tiles your way convert to terrain just fine, while getting 81, fewer, crashes iClone every time when I convert to terrain?

Tim
By TimothyMasters - 4 Years Ago
Arie - Thanks for that detailed response!

I did what you did and got exactly your result.  But now I have a mystery.
I selected all tiles and converted them to terrain, and it went perfectly.
Here's the mystery, and it's absolutely repeatable, not a one-time fluke
If I create a 9x9 in one shot I have 81 tiles, fewer than your 100.  But when I select them all and convert to terrain, iClone goes nuts (empty spaces, some in the scene list having physics, some not, etc.) and then it does a hard crash.

So... Why does getting 100 tiles your way convert to terrain just fine, while getting 81, fewer, crashes iClone every time when I convert to terrain?

Tim
By Kelleytoons - 4 Years Ago
If you have it working that's fine.

What I was talking about is if you select the master prop and choose "Merge Subprop" it will merge all sub props to the master, creating just one prop.
By pinguintje - 4 Years Ago
I think you also selected the box (Group) when you converted the planes to terrain.
Then you get this result.

https://forum.reallusion.com/uploads/images/b3c77e80-0df9-4d55-b1d7-6b0d.jpg

When you don't select the box (Group) and convert only the planes to terrain it will be ok and the box remains a prop. You can make this prop invisible.

https://forum.reallusion.com/uploads/images/e87f1a59-2db0-4088-b277-12b5.jpg

Wait with convert to terrain till you have finished the whole scene. (Once it's a terrain you can't change the terrain).
And then place the trees,flowers, props, characters etc.

Best Regards,
Arie

By TimothyMasters - 4 Years Ago
I did accidentally select the Group once and saw that weird result, but I learned my lesson!

I still do not have it working completely.  I had already put a LOT of work into creating a 9x9 tiling with perfect topography and I desperately want to get it converted to terrain.  I did make one major breakthrough when I discovered that it goes a lot better when I keep shutting down iClone and restarting.  In other word, I convert a bunch of the tiles to terrain, but then I do not convert another bunch, which will cause iClone to crash.  Rather, I save that intermediate project, shut down iClone, start it again, and load the intermediate project that I just saved.  Then I can convert a bunch more without it crashing.  I have to do that many times, one little bunch at a time, shutting down iClone and restarting it multiple times.  That got me 80 of my 81 (9x9) tiles converted from prop to terrain.  But there is one last tile that will just not convert.  Every time I try to convert it, iClone crashes.  I think I'm going to send it in as a bug report, which it surely must be.

If all else fails I'll use your 5x5 and copy method and start my topography creation all over again.

This is ridiculous.  I'm trying to create something, and instead I'm spending multiple days just trying to find work-arounds for iClone bugs.
By pinguintje - 4 Years Ago
I think it's not an iClone bug.
The conversion of 100 planes from prop to terrain should go in one time, but it takes a minute (or two).
Before you do the conversion first save the project.
If it doesn't work in one time select 20 planes and convert them. Wait till this is finished (you see it in the content manager), select another 20 and so on. But wait every time till it's finished, otherwise iClone crashes.
You don't have to start iClone every time again. 
I hope this will work.

With a few planes I have made this picture of Little Red Riding Hood and her friend the wolf.

https://forum.reallusion.com/uploads/images/ba3c7129-c442-4b56-ad63-3ef8.jpg 

Best Regards,
Arie

By TimothyMasters - 4 Years Ago
Arie - Have you really made something that large work?  I've tried everything imaginable to get a 9 by 9 with modest topology morphs to convert to terrain, on two different computers.

If I do one group and then another group when it finishes, iClone usually crashes (depending on the size of the groups).  But if I shut down iClone and restart it I can do groups like that.  But even then I can never finish the entire 9x9.

Here's another thing I tried, with high but vain hopes.  I selected all tiles, unlinked them from the Group, and deleted the group, just to get it out of the way in case it was part of the problem (I never tried to convert the group to a terrain!).
Then I selected all tiles except one and attached them all to that one excluded tile, creating a hierarchy, and then merged the hierarchy.  This got everything down to just one prop.  I tried to convert that one prop to terrain, but it rapidly hit 24 percent complete and then stuck there for an hour.  When I clicked the mouse on the screen, iClone did a hard crash.

I've tried lots of other things as well.  I even created a 3x3 and used your CTRL-Move trick to duplicate it up to 9x9.  That's not a great solution because then NTG's morph tools don't work right; they work right only when you create the whole thing in one shot.  But even that trick did not let me convert to terrain.

If you are so inclined, could you please try one simple thing for me.  Create a 9x9 at highest resolution, do some random topography to make some hills and valleys, and then convert to terrain.  If you can make it work, then there is something wrong at my end.  But it's two different computers, both with a lot of RAM, i9 CPUs, and high-end GPUs.  Thanks!




By Postfrosch - 4 Years Ago
Hello TimothyMasters,
something is wrong with you.
Here you have a 9x9 terrain (= 81 fields) - created with the Natural Terrain Generator.
Look this Pictures
Image 1: 81 fields created as props 
https://forum.reallusion.com/uploads/images/05bd3bfa-f641-4c9f-b413-766f.jpg
Image 2:  81 fields topology created (still props)
https://forum.reallusion.com/uploads/images/6675e6ff-cf39-48b4-bcdf-7a04.jpg
Image 3: 81 fields props converted into 81 fields terrain
https://forum.reallusion.com/uploads/images/0af18db6-0e5f-40f0-8ab0-500d.jpg
Creation - creating topology - converting to terrain approx. 8 minutes
IClone crashes:None

Mine conclusion: works flawlessly
Greets from Germany
Postfrosch


Annother 9x9 Terrain:
https://forum.reallusion.com/uploads/images/6ccd533b-154b-4606-9b45-9a76.png
By TimothyMasters - 4 Years Ago
Thanks for doing that.  Wow, how discouraging.  Both of my computers crash when trying this.  I can't imagine what's wrong.
By pinguintje - 4 Years Ago
@TimothyMasters

You see, it works.

The last thing you can try is:
-generate a terrain: 5x5, 32x32, 2 K
-create some hills
-convert it to terrain. That should take about 10-20 seconds.

Do no other things (merge etc.)
What happens? This should go easily.

@Postfrosch
Thanks for the explanation.
A very nice picture!!!:):):)

Best Regards,
Arie

By TimothyMasters - 4 Years Ago
I haven't found the exact cutoff, but I know that everything works perfectly at 5x5 max resolution, and fails at 9x9.  So there's a limit between 5 and 9.

I also learned that the size of the screen display makes a difference.  If I have the preview camera up close so the terrain mostly fills the screen, iClone crashes at 24 percent complete, but if I back off the camera so the terrain is just a tiny rectangle, it gets to almost 40 percent complete before iClone crashes.

It can't be a hardware or installation issue because the exact same thing happens on both my big tower that I use at home and the laptop that I use when traveling.

There are a million 'Preference' and 'Project setting' things that I just leave at their defaults because I have no idea what they do.  So maybe something is set wrong.  But I sure am getting tired of experimenting, because so far I've spent about 20 hours (seriously!) trying different experiments to try to figure out the problem.  The closest I've come is selecting a few tiles at a time and converting them.  That does not help if I just keep doing it over and over.  But it almost works if after each conversion I save the project, shut down iClone, restart it, load the project that I just saved, and do another block of tiles.  I can get almost all 81 tiles converted to terrain that way.

Talk about frustrating!
By Rampa - 4 Years Ago
Hi Timothy,

Try cleaning out your temp files. Start by quiting all running applications. Then try this:
1. Type %Temp% on the Search box on the Task Bar and click the %Temp% folder in the search list.
2. This will take you to C:\Users\<YOUR PROFILE NAME>\AppData\Local\Temp.
3. Highlight and delete all the folders and files in the Temp folder (some may remain, this is normal).
4. Go to C:\Windows\Temp
5. Highlight and delete all the folders and files there.
6. Restart your computer.
* NOTE: The Local\Temp folder is a hidden folder and so may not appear. To view hidden files and folders, do the following:
1. Open the Control Panel and select "Appearance & Personalization".
2. Click "Show Hidden File and Folders".
3. On the View tab, under Hidden files and folders, click Show hidden files and folders. 

By default, you iClone temp is at this location:
C:\Users\your name\AppData\Local\Temp

After making sure there is nothing left there, set yourself a custom temp location in the preferences. That way you can keep track if your filling up your drive with big temp files that take up all your working space. It makes deleting them, if you need to, super easy that way.
https://forum.reallusion.com/uploads/images/e096d0e0-a81b-4e71-9774-d05b.jpg

I created a folder on my desktop called "Reallusion Temp Files" and set it as the location in both iClone and CC3.

By TimothyMasters - 4 Years Ago
Thanks!  I'm going to try that now, and I'll let you know how it goes.

But I will say that I had a breakthrough last night that works in this case, though will not be a general solution.  I realized that almost 1/4 of my terrain will eventually be under water, so I figured out exactly which tiles will be hidden and I deleted them.  This got the count down from 81 to a little more than 60.  This was enough of a drop to let iClone successfully convert the remaining tiles to terrain in one shot.

That's when I got my next bad surprise.  Even with no other items except the water in the project, the frame rate plummeted to just a couple frames per second!

So then I did another trick that a kind and experienced user here had suggested in another context.  I went back to before the terrain conversion, when I had just the 60 prop tiles that would show.  I selected all but one and attached them all to the one not selected, creating a hierarchy.  Then I did "Merge sub-props' to get it down to just one prop.  I had no trouble converting this one huge prop to terrain, and now the frame rate is over 30 fps!  So I'm home free in this particular case, saved by the water.  I'll see how cleaning out TEMP goes, although I doubt if that's the solution, since I have hundreds of gigabytes of free space on my C drive.
By Kelleytoons - 4 Years Ago
Cleaning your folder out is unlikely to be the problem.

As I said before (at the start of this thread) it's an iClone bug/issue.  Anytime objects in a scene get over a few dozen it will slow down the system TREMDOUSLY.  Particularly running a Python script.  When you merged all the objects you eliminated that issue.

So - at the risk of repeating exactly what I wrote before - I've got more than a dozen scenes with hundreds of objects that bring my powerhouse system to its knees.  I take these exact same scenes, merge them into one prop SO IT HAS THE SAME TEXTURES AND POLYS and it works fine.  iClone just can't handle a lot of objects in a scene despite the memory you have.  Clearly there is some issue with maintaining pivots/object centers/identities/whatever.  Shouldn't be, really, but perhaps their tree walking is badly written code.

We need to get this fixed, if not in 7, then in 8 for sure.  While no 3D system is immune to slowdowns with large scenes, it shouldn't be the number of objects that kills the program.
By TimothyMasters - 4 Years Ago
That's my opinion as well, but unfortunately it's not the solution here, or probably not.  I say that because before my underwater deletion, I tried the 'merge sub-props' trick on the original 81-tile project.  That one huge prop would not convert to terrain, at least not in the hour I let it run before giving up.  The next time I need a full 9x9 scene, which will likely be very soon, I'll do the merge sub-prop on the full 81 and let it run overnight.  Maybe it will be done by morning.
By TimothyMasters - 4 Years Ago
I appear to have found a great solution to my problem of converting the output of a 9x9 to terrain.  It does not solve the ultimate problem of converting all 81 tiles to terrain; that still crashes iClone.  But as it turns out, I really don't want to do that anyway.  When I removed about 20 unneeded tiles to get the count down to around 60, the terrain conversion worked perfectly, but the frame rate was insanely low.  It would doubtless have been even worse if I had managed to convert all 81.

I had also given up on compressing all 81 tiles into a single huge tile, because that conversion seemed to hang iClone, followed by a hard crash.  But I accidentally discovered that the problem there was that I simply did not give it the enormous amount of time needed to complete, and if I did the tiniest thing like trying to move a window before it was done, iClone crashed.  But if I just go away and do something else, like maybe get a good night's sleep while it's crunching away, it eventually finishes correctly.  And as a bonus, the frame rate is excellent!  In summary, just to be clear, here is what I have so far found to work perfectly:

1) Select all tiles
2) In the Modify menu, unlink them from the Group.  This can take quite a while to complete.
3) Delete the Group
4) Select all tiles except one
5) In the Modify menu, attach the selected tiles to the one not selected
6) Under 'Edit' click Merge Sub-props.  This gives you a single prop that contains all original tiles.
7) Convert this prop to terrain.  WAIT FOR IT!  TOUCH NOTHING!

That last step is the critical one.  If you try to do anything while it is working, which takes a VERY long time, iClone will crash.

So far this has worked for me multiple times, so I'm confident that this is the way to go.
By Kelleytoons - 4 Years Ago
Exactly and it betrays the bug/problem with iClone - multiple objects (more than a few dozen) are too much for it.

This really is unacceptable.  If it's not fixed by 8 folks will leave the program for far better solutions.