Showing posts with label data cleanup. Show all posts
Showing posts with label data cleanup. Show all posts

2015-12-06

Another T5SS Data Drop: many new sectors In Review

The Vargr Extents and Julian Protectorate get some love in the latest T5SS data drop, with the following sectors newly "in review":
Also, way off on the Spinward edge of charted space, Group One's Theta Borealis – the last of the GDW "land grant" sectors – is now in review.


Some data corrections elsewhere as well. As always, check the sector data history on GitHub for the gruesome details, and provide feedback on the data on COTI.
Ω

2015-03-29

T5SS Data Drop - March 2015

DonM sent out a new T5SS data drop, adding Spica to the set of "in reviewed" sectors, and graduating the Aslan sectors like Riftspan Reaches, Ealiyasiyw and Dark Nebula out of review to "official".

Other tidbits from the drop:
  • Hive Federation now has T5SS allegiance code: HvFd
  • Embassies have have base code E
  • Base code X, previously reserved for Zhodani Relay Station, is slated to be replaced by W, which will indicate any Way Station, not just an Imperial Scout Way Station. That's not entirely reflected in the data yet.
Enjoy!

Ω

2014-05-20

T5SS Data Update - April/May 2014

The new data is now live!

Summary of changes:

  • T5SS sectors now have 4-character allegiances; these are visible on the map and in tab-delimited and column-delimited data files. The new allegiance codes are mapped to 2-character codes when "SEC" files are output or at low scales. Codes are documented at http://travellermap.com/doc/secondsurvey
  • T5SS sectors now have 1- or 2-character base codes, e.g. NS for Naval Base and Scout Base, when shown in tab-delimited and column-delimited files. They are mapped to 1-character codes when "SEC" files are output. Codes are documented at http://travellermap.com/doc/secondsurvey
  • Booklet and World Data Sheets now use the newfangled allegiance and base codes
  • The following sectors are under review as part of the T5SS. The data is updated to conform to T5SS formats, but the data is still subject to heavy change.
  • Tab-delimited files are now assumed to be UTF-8 (previously, was Windows-1252)
  • And, of course, nearly all of the sectors covered by the T5SS have had some tweaks.
This will doubtless have introduced issues visible on the site (e.g. borders and routes that need tweaking) and due to the extensive code changes it's likely bugs have crept in that will affect booklet/poster creation. Let me know here or via email and I'll try and get them fixed ASAP.

Ω

2013-12-29

Dotmaps/Placeholders

I recently introduced a trade code (Dotmap) - yes, parentheses included - for worlds where we only have position and allegiance information and no other system details, i.e. a dotmap, like those found in various DGP publications. If present, the world is rendered with an asterisk (*) for the world symbol, with no other details. This same symbology is used in Adventure 4: Leviathan for worlds in the Outrim Void that the players are expected to explore for fun and profit. This has been added to the map legend (in the new UI).

UPDATE: Support for (Dotmap) was dropped in June 2014. Use ? for unknown fields in UWP and PBG instead e.g. ???????-? ??? and see Foreven sector data for an example.

I implemented this for Luretiir!girr which I recently reconstructed from a dotmap

I also went back and imposed this on sectors I'd reconstructed from DGP's Megatraveller Alien: Solomani and Aslan: Ahkiweahi, Aktifao, Etakhasoa, Fahreahluis, Heakhafaw, Hfiywitir, Irlaftalea, Kefiykhta, Ohieraoi, Teahloarifu, Uistilrao, Yahehwe. Those sectors were totally randomly generated with no data checking, and I felt bad having them on the site. To make it clear that they were really uncharted and ready for someone else to detail, I removed the random data and marked the worlds as (Dotmap) in the data.

There are a few rough edges - I should filter these out in the data display and the booklet generator should add support for these.
Ω

2013-05-03

Data Updates

Just some data updates:

Ω

2013-03-31

New T5 Data Drop

Another T5 Second Survey data drop from Don McKinney. This one rolls in updated data for Ziafrplians and Gvurrdon, and fixes a few glitches like extra TL-G worlds that sharp-eyed users have noticed.

I've also rewritten parts of the page that display dynamic data - search results and the credits/links at the bottom - as well as the booklet generator to use Handlebars.js. I had been using JsonT but wanted to move to a more modern template system. Handlebars is derived from Mustache but supports paths which I needed. This should not have changed anything except for a slight performance improvement, but as usual let me know if I broke something.

Ω

2013-01-19

Dark Nebula cleanup help? - DONE!

To verify some world names (among other reasons), I retrieved my Traveller hardcopies from storage. My back is still recovering - dead trees are heavy!

Two of the pesky names were in Dark Nebula as published in DGP's Solomani and Aslan, and as suspected the names in the SEC files floating around are incorrect, due to transcription or OCR errors. I did a quick visual scan and it looks like as many as 10% of the world names have glitches (i/l or l/t confusion, primarily).

Does anyone want to help verify the names? I send you my SEC and scans of the UWP tables, you send me back a corrected SEC. All I can offer is fame and gratitude in return.

(Or you could sneakily rename a world after your cat or something.)

UPDATE: And thanks to help from Don McKinney, Dark Nebula has been updated. I'm not sure where the file I had originated, but L.W.L.Guatney transcribed a corrected one back in 1993. Andrew Brandford had also volunteered to help, so he also deserves the community's thanks.

Ω

2012-11-18

T5 Second Survey Data

As part of an ongoing development process for Traveller 5 the data set for all worlds in the Imperium are getting reviewed. With help from Don McKinney, the latest data is now live on the site. Thanks to everyone who's been contributing nitpicks and reporting errors - this information has been fed back into the process to help make sure the new data set is as high quality as possible.

The following sectors have been updated, baselined at 1105:
In addition, the following tweaks have been made:
  • Added "Reavers' Deep" and "Reavers Deep" as aliases
  • IFRAME no longer steals focus (which fixes the API page to not scroll down on load)
  • Speaking of the API page, it now links to the correct AotI coverage area
  • Fix Egyrn Subsector's name (thanks to Makoto MACHIDA for commenting)
Many of the data changes will be intentional, but obviously with this volume of cleanup there will be further glitches, so start the bug reports coming!

Ω

2012-09-03

Data Updates

I've finally had a bit of time to make a minor dent in the backlog!

  • Corrected spelling of "Capitol" => "Capital" in Far Frontiers sector remarks so that Harantz and Alzenei are properly highlighted (c/o 77topaz)
  • Updated data file for Empty Quarter (c/o Alvin Plummer and Don McKinney)
  • Added subsector names for Gzaefueg (c/o Alvin Plummer)
  • Added subsector name for Ilelish subsector O: Zagrab (from the Zhodani Base, c/o Alvin Plummer)
  • Credited Greg Videll (HIWG) for Zarushagar sector work
  • Corrected Corridor 2006 Tamilaa's base "G" => "N" per Atlas of the Imperium (c/o Dan Holmberg)
  • Renamed Gelath/Gn'hk'r 0704 "X!'kib'grikr" => "Aat Nokr K'ng" per GT:AR2 (c/o Alvin Plummer)
  • Fixed names of Dagudashaag 0236 "Kinhe" => "Feym'n", 0739 "Hanas Soujourn" => "Hana's Soujourn" and 0840 "Uusjar" => "Uushar" per Signal GK #11 (c/o Art Gorski)
  • Fixed name of Reaver's Deep 1224 "Roikhoi" => "Roakhoi" per the Reaver's Deep Sector Sourcebook (c/o Dan Holmberg)
There's a lot more to do, of course, in terms of data corrections and feature work. And, of course, overall site stability.

Also, the LBB generator ("sector.htm") now uses a more print-friendly layout and should automagically print in landscape mode with correct margins in modern browsers. (It will look best if you have the Optima font available on your system.)

NOTE: Some of the search indexes don't appear to have updated for the above changes. I'm investigating.
Ω

2012-03-19

Julian Protectorate Routes

More data cleanup happening - see the comments thread at http://travellermap.blogspot.com/2012/02/data-corrections.html:


Mike Kuehn submitted trade routes for the Julian Protectorate - that's Amdukan, Mendan, Ingukrax and Star's End sectors, with a few tweaks in Antares and Empty Quarter. Ω

2012-03-14

More Data Cleanup

Thanks to the sharp eyes of commenters like 77topaz and research by Swiftbrook and others, several more data glitches have been identified and corrected:

  • Gateway Domain - Gateway, Crucis Margin, Glimmerdrift Reaches, Ley Sector - checked against the Gateway to Destiny: The Gateway Domain Sourcebook, name truncations fixed
  • Far Frontiers - polity capitals are now highlighted
  • Ilelish - truncations fixed (guesswork)
  • Yiklerzdanzh - truncations fixed (present in source SEC file as an appendix)
  • Spica - bracketing characters in names removed (were these formatting codes for some program?)
  • Vanguard Reaches - truncations fixed - sector file was regenerated from Chuck Kallenbach II's source data 
There are a few more things that need fixing, and certainly more sectors could use a once-over. As an administrative note, data glitches like repeating names or bogus UWPs will not be corrected on a world-by-world basis for any of these regions:
All of these vast regions are, at best, placeholders. Should you wish to "claim" a sector and give it a thorough scrub and provide background, go for it.
Ω

2012-02-25

Spinward Marches 1105

The Spinward Marches has been updated (backdated?) to 1105 to match the other data updates, bringing it in line with the T5 Spinward Marches poster by FFE. The updated data is courtesy of Marc Miller, Donald McKinney and Robert Eaglestone. This includes the sector data and metadata (routes and borders).

In addition, where the "Sector Data" link was at the bottom of the map has been replaced by two clearer links: "Source" which takes you to the published source of the data (if any), and "Data", which serves up the SEC data used by the site itself.

I'm sad to report that, as part of this cleanup, the anomalous jump-5 X-boat route between Tenalphi and Strouden that's been with us since Supplement 3 is no more. It's been replaced by a jump-4 route between Tenalphi and Persephone. Gone are the days of envisioning it as everything from an experimental route to a spatial anomaly. At least in the OTU...

Ω

2012-02-22

Data Corrections

Thanks to Don McKinney, I have updated data for the following sectors:
In addition, I made some individual world tweaks:
The truncated names appeared to be a result of passing the data through a fixed width data handler at some point before the files came to me. It's possible there are others - reports welcome!
Ω

2011-11-01

Metadata Tweaks


Some minor cleanup:


Ω

2011-09-12

Deneb, Ziafrplians, SectorMetaData, bug fixes

A few recent updates, tweaks and fixes:
  • Updated data for Ziafrplians from Don McKinney. This is the same data as used for the Zhodani Alien Module recently published by Mongoose, now a canonical part of the OTU. Check out the book for many more details about the sector, of course.
  • Updated data for Deneb by Robert Eaglestone, including a few X-boat route tweaks. This is part of the ongoing T5 data cleanup project. Expect the rest of the Domain of Deneb to follow at some point soon.
  • A new SectorMetaData API
  • Tweaked Candy Style rendering. Now includes UWP at high scales, and scales a little more nicely.
  • Fixed behavior of the experimental touch version when moving from "pinch" to "drag" (i.e. letting go of one finger while the other is still touching)
Ω

2011-01-27

Allegiance Fixes

Minor tweaks:

Ω

2009-03-28

Stellar Data

A week or so ago I slipped out a file format page, adapting Marc Miller's article in Challenge #26 describing the first standard sector data file format. It's pretty bare bones at the moment - just what's in the article so it is preserved for posterity - but I plan to grow this over time to document other sector data file formats that have appeared over the years. Call this "Version 1.0".

(Honestly, posting it was inspired by site users wondering what the various base codes in the data meant. But everyone seems to love file format discussions...)

Then, inspired by Yet Another Thread on sector data file formats on CotI I decided to finally tackle stellar data.

  • Stellar data now output by SEC.aspx API when present in source data. The following grammar is used:

// Basic:
// system ::= star ( w+ star )*
//
// Extended: (Malenfant's Revised Stellar Generation Rules)
//
// system ::= unit ( w+ companion )*
// companion ::= near | far
// near ::= unit
// far ::= "[" system "]"
// unit ::= star | pair
// pair ::= "(" star w+ star ")"
//
// star ::= type tenths w* size main?
// | dwarf
// | unknown
// type ::= "O" | "B" | "A" | "F" | "G" | "K" | "M"
// tenths ::= "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" | "8" | "9"
// size ::= "D" | "Ia" | "Ib" | "II" | "III" | "IV" | "V" | "VI" | "VII"
// dwarf ::= "DB" | "DA" | "DF" | "DG" | "DK" | "DM" | "D"
// unknown ::= "Un"
//
// main ::= "*"
//
// w ::= " "


  • "Un" in stellar data is from Mendan 0221 in Challenge #46 (by Mike Mikesh). That could be a typo, but I'm interpreting it as "Unknown" (mysteries abound!). Note that the world is an Amber zone, so it's a navigational hazard at least!
  • Like other SEC.aspx output, the data is normalized - that is, it's been parsed and then re-output. In some cases this may filter the data. You get what you pay for. In particular, although Malenfant's revised stellar generation data is now consumed by my site, only basic star data is output. Anyone have a link to Malenfant's rules?
  • SEC.aspx output now has a sector header, similar to the GEnie data. The field lengths are NOT guaranteed to remain fixed, however. Consumers that rely on fixed width fields (rather than Regular Expression or Grammar-based parsers) should parse this header to determine the field lengths.

1-25: Name
27-30: HexNbr
32-40: UWP
42: Bases
44-68: Codes & Comments
70: Zone
72-74: PBG
76-77: Allegiance
79-98: Stellar Data


  • I tidied up some bogus stellar data: Night (Core 0839) "K8 VI M4" to "K8 VI M4 D" (presumably a truncation); and Tender Mercy (Ley 0314) "2 V A6 D" to "K2 D M6 D" (ditto); in both cases I used older GEnie data as the source
  • While I was there, I fixed a couple of O/0 (oh/zero) typos: Kaskar (Vland 1933) stellar "MO V K6 D" to "M0 V K6 D", and Zdienjbats (Far Frontiers 1801) PBG "BO4" to "B04". The latter actually inhibited the world from showing up on the map - oops!
That's it for now! Ω

2008-01-08

Updated Data

A sharp eyed user noticed that the following worlds were missing:
Some digging revealed that in some cases the data I had which was indicated as being sourced from the GEnie upload was, in fact, post-processed and had lost some data. I wrote some tools to look for other inconsistencies and found one more Dark Nebula world that had gone missing.

After some soul searching, I decided to pull in updates to the data which had been made by a project led by Robert Eaglestone and Jim Fetters on the COTI board in 2005/2006 which analyzed the parts of the Atlas of the Imperium containing Imperial worlds as a run-up to scrubbed data for T5. This resulted in updated data for:
  • Alpha Crucis
  • Antares
  • Core
  • Corridor
  • Dagudashaag
  • Daibei
  • Delphi
  • Diaspora
  • Fornast
  • Gushemege
  • Ilelish
  • Lishun
  • Magyar
  • Massilia
  • Old Expanses
  • Reaver's Deep
  • Verge
  • Vland
  • Zarushagar
(Empty Quarter was also updated, but I kept the Stellar Reaches CT version.)

In theory, this process deleted 5 worlds but added 12. Let me know if you notice any new AotI inconsistencies.

[Update 2008-01-09]
The changes are:

No, I can't count - that's +9 -4, not +12 -5. The difference is the Empty Quarter (+3 -1) that I didn't incorporate. Ω

2006-10-19

Vargr Allegiances and Rimward States

Done! I've now generated borders for every sector. Of course, some still need tweaking... but I think I'll take a break now.

Most recent changes:
  • Added borders for: Ohieraoi, Fahreahluis, Hfiywitir, Irlaftalea, Teahloarifu, Ahkiweahi', Iyiyukhtoi', Banners, Hanstone, Malorn, Hadji
  • Added allegiance codes from Vilani and Vargr
  • Show labels for regions even if they're standard allegiances (e.g. "Im", "Zh", etc)
  • Added labels for J-4 and J-5 rift spanning routes (micro-scale)
  • Added allegiance code for Regency of Muirimi (Theron and Iphigenaia)
  • Corrected allegiances of Eha and Yuhoahtiyol (Hanstone Sector) to match Solomani and Aslan
Oh, and I wanted to share this quote about the Julian Protectorate:

"The borders of the Julian Protectorate are not easy to define..." - Michael R. Mikesh, Challenge #49

No kidding!

Take a peek at how I did it, and let me know what you think. Ω

2006-10-12

Data Cleanup and K'kree/Hiver borders

A few more borders in Hiver and K'kree space:

...and then some data cleanup that I realized was necessary when I got there!

I noticed in some of the “outlying” sectors (i.e. non GEnie data) that the UWP parser was misinterpreting some worlds (e.g. thinking the PBG was a trade code, the allegiance was PBG, and the stellar data was the allegiance) or missing some worlds entirely. This is because when I started I used a very loose pattern matching system to try and incorporate lots of different data sets without editing the sector data itself. This turns out to be intractable, so I’ve given up and tweaked the data in some sectors to be a little less quirky, and tightened up the parser. The site should also be marginally faster in loading data (i.e. if no-one has viewed a region in a few seconds).

It's possible that a few worlds have now suddenly popped into existence elsewhere. If you notice borders that are shy a few parsecs of a world, let me know!

Data tweaks:

While I was there, I wrote some tools to validate the data and corrected some simple typos or missing digits in other sectors:

Ω