/go/ links and history

Two tidbits:

First, I added support for /go/ links, which can take the form:

http://travellermap.com/go/sector - e.g. http://travellermap.com/go/deneb

or

http://travellermap.com/go/sector/hex e.g. http://travellermap.com/go/deneb/0406

These are really shortcuts for the ?sector=sector&hex=hex form, but should be easier to type.

Any other such shortcuts you'd like to see?

Second, for those who are bemoaning changes to any of the data, just a reminder that all changes are tracked on Github at https://github.com/inexorabletash/travellermap/commits/master/res/Sectors - you can drill further into a particular file and see the changes over time.

Ω

Foreven - placeholders or not?

Should I replace the Foreven data that I'm using (by James "GypsyComet" Kundert, Dylan Lee and P-O "BeRKA" Bergstedt, original here) with placeholders per the Foreven Sector Reserve doc?

I'm leaning towards "yes". What say y'all?


Ω

Dotmaps/Placeholders

I recently introduced a trade code (Dotmap) - yes, parentheses included - for worlds where we only have position and allegiance information and no other system details, i.e. a dotmap, like those found in various DGP publications. If present, the world is rendered with an asterisk (*) for the world symbol, with no other details. This same symbology is used in Adventure 4: Leviathan for worlds in the Outrim Void that the players are expected to explore for fun and profit. This has been added to the map legend (in the new UI).

UPDATE: Support for (Dotmap) was dropped in June 2014. Use ? for unknown fields in UWP and PBG instead e.g. ???????-? ??? and see Foreven sector data for an example.

I implemented this for Luretiir!girr which I recently reconstructed from a dotmap. 

I also went back and imposed this on sectors I'd reconstructed from DGP's Megatraveller Alien: Solomani and Aslan: Ahkiweahi, Aktifao, Etakhasoa, Fahreahluis, Heakhafaw, Hfiywitir, Irlaftalea, Kefiykhta, Ohieraoi, Teahloarifu, Uistilrao, Yahehwe. Those sectors were totally randomly generated with no data checking, and I felt bad having them on the site. To make it clear that they were really uncharted and ready for someone else to detail, I removed the random data and marked the worlds as (Dotmap) in the data.

There are a few rough edges - I should filter these out in the data display and the booklet generator should add support for these.

Ω

Poster Maker

The old post.htm and jpost.htm pages were really API documentation about the poster and jumpmap services, with built-in demos that could be used to generate images. But they were decidedly non-friendly.

To go with the new UI on the main page (still in test!) I've spun up a new "Poster Maker" page that aims to be user-friendly. It handles sector, subsector and jumpmap images, and offers a "quick preview" mode. It also defaults to PDF since that's the best format to print if you don't know what you're doing.

Try it out, send me feedback!

http://travellermap.com/poster.html

(The "options" box rather obviously needs to be replaced with a slew of checkboxes for the individual knobs. I'll get to that when I can.) - done!

Ω

Zoom

I made a bit of a video:


I wrote a Python script which generated Tile API URLs centered on Regina at various scales. Piped that through to curl to fill a directory with images, then used ffmpeg to assemble them. Once I had the kinks worked out it only took about 10 minutes to render.

(I used the live site rather than my development machine. If you attempt something similar, be gentle and don't request more than one image at a time!)

Ω

Luretiir!girr

I found an old dotmap, possibly by L. W. L. Guatney, that included a portion of K'kree space including one sector that no-one has tackled yet - Luretiir!girr.

I've generated a dotmap sec from it (i.e. only world positions, worlds are all nameless and X100000-0), and drew borders / picked allegiances based on the Imperium Map poster. This is totally unofficial but I thought it was worth preserving the dotmap data in case anyone decides to tackle the sector in the future.

I arbitrarily decided that the large polity jammed into the Spinward side of the Two Thousand Worlds was the Dominate of the Lords of Thunder, but it will be up to whoever officially defines the sector to identify the local governments and decide which, if any, is the Dominate. Ω

Button Labels

On the New UI I use graphical labels for some of the buttons - a magnifying glass for search, a question mark for help, a gear for settings, and a house for "home". Those are all pretty universal iconography for the Web at this point, and there are tooltips for accessibility.

I'm using textual labels for three buttons:

• Legend - this is pretty unique to this site, and I can't think of a universally understandable icon. I'm pondering whether Legend deserves to be a top level item or should be under "?". I also hide Legend on small screens.
• Important - this is a "default" search for important worlds.  Again, no universal icon for this concept. I believe this feature will evolve into more of a "browsable, sharable sets of points of interest". Maybe a point-of-interest icon, then?
• Share - this one is starting to shake out as a standard icon - the three circles in a < formation - but it's not quite universal yet.

Thoughts?

Ω

Nothing to see here, I hope...

I've rejiggered a bunch of the internal workings of the site in ways that are only of interest to developers. The last 40 or so revisions moved away from serving data and images using "ASPX" pages to HttpHandler. Ideally, there's no change in behavior except that unnecessary cruft isn't running. The best part from a developer perspective is that all URL handling for the services is now done programmatically - no hosted files are necessary to add APIs, just code. The code for that lives in the Global.asax.cs file, if you're curious. As an added bonus, I'm using regular expressions for URL dispatching so it's easier to e.g. distinguish /data/sector/subsector from /data/sector/hex and more possibilities are now available - I should be able to enable named subsector references, for example.

Since there are third party sites and services using the old "aspx" APIs I've carefully added aliases to allow those to continue to work. If you're using them, please try and migrate to the new APIs (/api/...) ASAP.

And of course, if I broke anything, please let me know. I fixed one bug along the way - JumpMaps were not correctly eliding out-sector borders. I can't believe no-one told me about that!




Ω

New UI Design

I've redesigned the UI for the main map page. Please try it out:

http://travellermap.com/new.html

I've tested in Chrome, Firefox, Safari, and Internet Explorer 8/9/10, and on iPhone and iPad - yes, you now have full UI control on touch-based devices.

Please let me know if you have any issues with the page. If you notice any glitches, it'd be nice if you check with the "classic" page as well to see if you've discovered a long-standing bug or something new with the UI.

Let's shake those bugs out!

Ω

Recent Changes

Quick update on some minor changes that have gone in recently:

• Added Holowan / Rim Worlds sector, by Brian Smaller
• Added Mikhail / Varan sector by Ed "Dalthor Et Magera" Anderson
• Added subsector names for Knoellighz, Dhuerorrg, Ngathksirz sectors, by Mike Davis - thanks to Dale Jenkins for the pointer.
• The credits area will display "Unofficial" if the sector data is not vetted data from the T5 Second Survey. Foreven gets a special "Preserve" indicator.
• The old subsector.htm demo page (a prelude to the LBB generator) has apparently been broken for months and no-one noticed, so it's been deleted.
• Sector posters now suppress the subsector grid lines around the edges, which were just visual clutter.

Ω

Highlighting Official vs. Unofficial Data

I'm starting to get a slow but steady stream of feedback in various forms that highlights an issue: TravellerMap.com is being treated as an authoritative site for the OTU, yet it presents a mix of official and unofficial data and doesn't clearly deliniate between the two.

When I started the site (2005) there had been a lull in Traveller. I believe Marc was just getting FFE together. There was no official data available in digital form. Online data sets blended data from CT/MT /TNE/T4 supplements, GEnie and other semi-official uploads, edits and additions by individuals and groups such as HIWG, and various large and small efforts to fill in the gaps, some intricately detailed and some large swaths of random die rolls. I completely admit that I went for "quantity" over "quality" since having a universe to zoom and pan around in was fun!

Now we have an official data set (the Traveller 5 Second Survey) and I'm honored to be able to host it on the site. It's blended rather seamlessly with unofficial data, and that leads to unfortunate confusion. Folks will ping Marc asking questions about sector data that's unvetted submissions of dubious ancestry.

IMHO, it's pretty high priority for the site to make some distinction here, but I'm not sure what approach to take. Here are some ideas:

• Take advantage of the "credits" at the bottom, and highlight "official" data more clearly with an indicator/graphic of some sort.
• Contrariwise, assume "official" is the default; highlight "unofficial" data
• Both of the above - always clearly show the provenance of the current focus
• Offer an option to filter unofficial data e.g. another checkbox on the side that would not even render unofficial sectors on the map.
• As above but have the option enabled by default (i.e. make unofficial data "opt in")
• Alter the rendering for "unofficial" regions - e.g. shaded background, dim everything slightly, etc.

These all have trade-offs between being too subtle or two imposing. I posit that as primarily an entertainment resource, most users enjoy seeing the "gaps" filled in, and it's only "serious researchers" who want to filter out anything unofficial. Of course, the credits and associated documents already highlight the sources of the data so I may be trying to optimize for a small set of users who are taking the site as authoritative but not actually reading the manual, which might be a waste of time.

On the other hand, I think it's a disservice to Marc and others if the hard work of the T5SS is lumped in with some randomly generated cruft I scraped off the web. On the third hand, many of the unofficial submissions have been labors of love and I'm just as honored to host them in all their unofficial glory (with both the authors and myself knowing full well they may get paved over at any point.)

Relevant stats: not counting the Zho Core Route, there are 154 sectors with data, 30 of which have "official" data.

My gut reaction at this point is to do something quick and cutesy like have == AMBER ZONE: UNOFFICIAL DATA == appear in the credits area that links to an explanation of the site's combination of official + unofficial data. I don't know if that's too subtle or too obnoxious. I'm also working to get some more obvious "official" / "unofficial" tag into the data returned by various APIs.

I welcome any thoughts and ideas for how to address this issue. Comment away! Ω

Data Format Documentation

I've posted two documents. Please take a look and suggest improvements/corrections:

File Formats - This document is intended to discuss the nitty gritty of file formats. It starts with a recap of the format published in Challenge #26 and used GDW's TRADER application for the Apple II, attempts to document the format used by Joe Fugate for the GEnie data upload, loosely specify the "sec" files found on the Internet, and define two canonical T5 data file formats, as used by the site. We'll see if they catch on.

Second Survey Data - This recaps the definition of the fields describing a world. For many of the fields this is covered in the Traveller 5 Core Rules but this document goes into more detail on some specifics hidden in the data (like sophont populations) and legacy fields.

I aim for these to be "live specifications" - that is, updated over time as new formats emerge. Right now they try to balance being rather formal specifications with being readable by non-programmers. We'll see how things go.








Ω

It's here

The stuff I promised in my last post is now live.

• Cleaned up API URLs and "semantic" URLs
• You can now POST MSEC metadata to the poster/jumpmap APIs 
• A blather of additional support for T5 data.

And perhaps least interesting if you're a user of the site - unless I get run over by a cable car or something - is that the source code to travellermap.com is now up on github.

Also:

• Slipped in a few bug fixes. Probably slipped in a few bugs, too. Let me know.
• Added a ton more tests (unit tests, API tests).
• I still need to post T5 data format documentation. (Writing good documentation is much harder and time consuming than writing code.)
• The old-style SEC data output has been changed to match the column widths from the ancient GEnie data drop. This means names get truncated and all that fun stuff. If this breaks anything that consumes the data I would recommend changing to consume the tab-delimited data, which is inherently more flexible.

Ω

Coming soon: Open Source, T5 Data, RESTful URLs, MSEC parsing

I'm taking a week off from work to catch up on personal projects that require more than the usual 20 minutes a day of unbroken concentration I can scrounge up. For TravellerMap.com my big goal is getting the code open sourced, up on GitHub. As a prelude to that I needed to do some cleanup, such as moving some authentication strings out of the code itself, and general polishing. Don't expect it to be beautiful, though!

A few things got done as part of this process:

• More RESTful URLs will be added, with the ugly SomeAPI.aspx URLs relegated to legacy. For a hint of what this might look like (if you're not familiar with the terminology) look at a COTI thread on RESTful Traveller APIs. 
• Nit: it won't be truly RESTful as the site is stateless, so there's no state to transfer. 
• I've implemented T5 "Second Survey" data parsing and a formal format description. Short answer: column-based, but variable width fields with a header that self-documents field widths.
• I've implemented support for sec2pdf's MSEC file format. Not every feature (like label tweaks) is supported, but it will save you from learning the XML metadata format.
• Added honest-to-goodness unit tests for a handful of things. 
• Converted most of the C# to use System.Linq where possible to make the code more readable.
• Squashed various bugs

Expect shiny things over the next few days, and as always forgive the mess.

Ω

Traveller5 Data

You may notice Traveller 5 Second Survey data starting to appear in various places in the site. It's going to be a gradual process to get it everywhere it makes sense. You can track the TODO list on the Trello board.

Where it shows up now:

• SEC API - if you add type=SecondSurvey, e.g.: http://travellermap.com/SEC.aspx?sector=spin&type=SecondSurvey - this is human readable.
• SEC API - if you add type=TabDelimited, e.g.: http://travellermap.com/SEC.aspx?sector=spin&type=TabDelimited - this is easier to parse
• "Data" link at the bottom of the main map page (which uses SecondSurvey, above)
• LBB generator (sector name link at the bottom of the main page) e.g. http://travellermap.com/sector.htm?sector=spin

In addition, I've clarified the credits area to distinguish between the source of the data (i.e. Traveller 5 Second Survey) from any products published by a current licensee (e.g. The Spinward Marches by Mongoose Publishing).

I still need to do something with the data in these parts of the site:

• Search integration - allow search by extension fields
• Display - show indicators for new data when zoomed in
• Document - explain the new T5 fields and data formats
• Accept T5 data in the post.htm / jpost.htm forms

Ω

Hive Federation Completed

Mark Humphreys has submitted the sector data, borders and routes for the last of the hitherto uncharted sectors of Hiver space, as well as updates to the other sectors, so it's all filled in on the map:

• Avereguar - new!
• Blaskon - new!
• Centrax
• Folgore - new!
• Gzirr!k'l
• Kidunal
• K'trekreer - new!
• Nooq - new!
• Porlock
• Ricenden
• Wrenton

Awesome work, Mark!

Ω

Nits

Aslan Tlaukhu (or however you spell it) bloc allegiances for A0-A9 should show up in SEC output now.

Also, I made a tweak to how "render slop" works. This will either improve performance/stability a bit, or lead to missing data in certain tiles. Let me know...

Ω

Data Updates

Just some data updates:

• c/o Mark Humphreys, updated data for Ricenden, Gzirr!k'l, and Centrax, and new data/metadata for Wrenton and Porlock in Hiver space
• c/o the newly relocated Traveller Wiki, added subsector names for Itvikiastaf and Tuglikki

Ω

Performance Fixes

To try and improve the stability and responsiveness of the site (sill hovering at about 99.6% which is okay but not great) I grabbed a trial copy of the ANTS Performance Profiler and did some CPU and Memory profiling. Good news - there was some low-hanging fruit that may explain some of the issues. All of it came down to non-obvious behavior in the PDFsharp library I use as an intermediary for rendering to bitmaps and PDFs.

• GPU: Each transform applied to the graphics state resulted in two matrix multiplies. In many cases my code was stacking up to 5 transforms before rendering anything (scale, rotate, translate, rotate again, scale again...). This showed up as a CPU hot-spot. This was greatly reduced by composing the matrices locally then sending the final matrix into the graphics context.
• Memory: Every text operation (DrawString, MeasureString) resulted in the allocation of several temporary StringFormat objects. Just a few seconds of panning around the map would generate 3MB of such objects - even more than temporary strings and at least an order of magnitude more than any other memory churn. They would all be GC'd eventually, but under load this could have caused the site to blow past the application memory pool limit resulting in a reset. This was fixed by changing PDFsharp to re-use the default objects and I reported the issue to the maintainers.

The remaining major CPU hot spot is, unsurprisingly, in clip path intersections for borders and sector boundaries, which is probably the most subtle and complex part of the map rendering process. Unfortunately, it will be hard to optimize directly but I can probably reduce the need to do it in the first place, e.g. more aggressively pruning which borders to render for a tile, and only doing sector edge clipping if there's a border that goes outside the sector in the first place.

I'm hoping that above fixes will result in higher site availability - I'll be watching my monitoring service (Pingdom) to see if they've made a material difference.

By the way, the ANTS profiling tools are quite nice. There's one for Memory and one for CPU. Like most profilers they work by attaching to your process and recording what functions are taking time (for CPU) or what objects are being allocated/freed (Memory). The ANTS tools integrate very nicely with ASP.NET - I was able to just point them at my development directory and say "go" and they launched a dedicated web server and browser so I could generate some traffic. The results are also really easy to understand - there's the traditional drill-in display that shows you inclusive time for each function, but also a source display and interactive call/allocation graph. Very effective!

UPDATE (2013-04-07):

Just to show how helpful profiling tools can be: I was investigating the border clip path logic and found a flaw in the "selector" code which helps determine which sectors are "in view" when rendering a tile. It was under-computing the number of sectors overlapped by a tile. Fortunately, there's already a 1-sector "slop factor" that's necessary so that e.g. routes or labels that cut across a sector's boundary are rendered, so this bug never manifested as missing content. Unfortunately, the correct code meant that even more time was spent on border path clipping (c/o the ANTS CPU profiler).

Borders are a special case where they are always clipped to the sector bounds anyway, though, so I rather than relying on the expensive clipping operations to do the right thing I added a simple bounds-intersection test. This cut border rendering from nearly 55% of the time of a typical tile rendering pass to only about 15%, under the overhead for drawing worlds (about 22%). This should also help with site stability.

Ω

New T5 Data Drop

Another T5 Second Survey data drop from Don McKinney. This one rolls in updated data for Ziafrplians and Gvurrdon, and fixes a few glitches like extra TL-G worlds that sharp-eyed users have noticed.

I've also rewritten parts of the page that display dynamic data - search results and the credits/links at the bottom - as well as the booklet generator to use Handlebars.js. I had been using JsonT but wanted to move to a more modern template system. Handlebars is derived from Mustache but supports paths which I needed. This should not have changed anything except for a slight performance improvement, but as usual let me know if I broke something.

Ω

Labels, Retina, and Theta Borealis

Some minor but shiny stuff:

• Display updates and animations now use requestAnimationFrame if supported, and I used Chrome's developer tools to look for hot spots and removed some inadvertent forced layouts in loops, which should mean lower CPU load and "reduced jank" when panning the map.
• Thanks to 77topaz for pointing out that I'd broken the map in older versions of IE. A quick fix to my requestAnimationFrame polyfill and the map should be happy in IE9 again.
• I tweaked the presentation of polity labels seen when you first load the page - they wrap so they're a little more readable now. Sector names now wrap too, which more closely matches the Imperium Map poster. I was also able to bump up the font size. In summary: the initial view of the map should be prettier.
• If you view the map using a modern browser that supports devicePixelRatio on a device with a high-density ("retina") display, such as an iPhone 4S/5, iPad 3, newer MacBook, Android, or even the shiny Chromebook "Pixel", the tiles are now rendered with higher quality. Summary: more pretty!
• On a retro kick, I swapped in the data, borders and routes from Group One's Theta Borealis Sector supplement. Watch out for the Zerp! Note that the double worlds in the original's hexes 2901 and 2118 are not shown on the map and won't appear in the data.



Original - Group One	TravellerMap.com

Ω

World Name Rendering

I was in a brick-and-mortar game store today and flipped through one of the Mongoose Solomani books. After noticing some glitches in the map (rendered using this here site!) I got off my duff and made a long-awaited tweak that ensures that world names are always rendered on top of nearby hex's zone indicators. It's a minor detail but should improve the situation in a couple of places.

On the "TODO" list but not yet done is ensuring that glyphs always have a background; some of them (e.g. Research Station, Prison) are difficult to see if the world is also a red zone.

Ω

Get your Hiver on

Thanks to exploration work by Mark Humphreys, three new sectors in Hiver space have come online including world data, borders and routes:

• Gzirr!k'l - straddling the Two Thousand Worlds and the Federation with a buffer zone of client states between. This sector bore the brunt of the Hiver/K'kree War in ages past, and bears the lingering scars.
• Ricenden - home to Guaran, the Hiver homeworld
• Centrax - home to Glea, the Federation capital

These aren't the final data or metadata, so expect some refinement over the coming months, and perhaps a few new sectors nearby.

(This is all unofficial data, of course. That will be obvious to long time visitors, but I'm going to try to be more explicit going forward given that I'm hosting both official and unofficial data without much distinction.)

Ω

Site Offline for Server Upgrade - COMPLETED

I've filed a request with my service provider to update the server to a new version of the HTTP server and OS. The site will be offline for up to a few hours (barring anything catastrophic).

As Sam (the man) Jackson says in such circumstances, "Hold on to your butts!"

...

Update:  4:47PM PDT

The site is up but has moved to a new IP address. It may take several hours for DNS caches across the 'net to resolve to the new address. In the mean time you can access the site via:

http://travellerma.web707.discountasp.net

...

Update: 5:28PM PDT

At least for me, http://travellermap.com is resolving. It may take several more hours for that address to resume working for you, so use the temporary URL above if necessary.

Ω

Data Overview 2013

Back in 2009 I promised a yearly update with what was new - data, metadata, and so forth. I suck at remembering to do this, apparently. Someone remind me in 2014, okay?

So here's the update for what's new since the last state-of-the-Imperium update:

• Filled borders make the map just that much prettier and identify polities
• Details for a few more sectors around the edges of the rift, c/o 77topaz
• Routes for the Julian Protectorate, c/o Mike Kuehn
• Routes for Alpha Crucis, c/o David Pulver

And the biggest set of changes, although they aren't really visible at this level:

• Scrubbed T5 data for Imperial sectors, c/o Don McKinney and Marc Miller

Previously

• Data Overview 2009
• Data Overview 2008
• Data Overview 2006

Ω

Dark Nebula cleanup help? - DONE!

To verify some world names (among other reasons), I retrieved my Traveller hardcopies from storage. My back is still recovering - dead trees are heavy!

Two of the pesky names were in Dark Nebula as published in DGP's Solomani and Aslan, and as suspected the names in the SEC files floating around are incorrect, due to transcription or OCR errors. I did a quick visual scan and it looks like as many as 10% of the world names have glitches (i/l or l/t confusion, primarily).

Does anyone want to help verify the names? I send you my SEC and scans of the UWP tables, you send me back a corrected SEC. All I can offer is fame and gratitude in return.

(Or you could sneakily rename a world after your cat or something.)

UPDATE: And thanks to help from Don McKinney, Dark Nebula has been updated. I'm not sure where the file I had originated, but L.W.L.Guatney transcribed a corrected one back in 1993. Andrew Brandford had also volunteered to help, so he also deserves the community's thanks.

Ω