Topic: MSP — MarySue Protocol [Draft version]

[Etarna]

MarySue finally got me thinking.

This is an open invite for authors of MarySue, ImmersionRP and any other active RP mod authors to weigh in on a new, common whisper-based protocol. Hopefully, with appropriate throttling and gentleness, despite the poor scalability, it could work better out on the servers than xtensionxtooltip2 does.

My first proposal is simple. Let's make the protocol, which is really _really_ simple in MarySue v1, extensible so we can all use it, and more efficient, by simply naming the fields and not requesting the profile (description) unless we want it.

Warning: All field contents, as all strings in WoW, are UTF-8. This means multi-byte characters sometimes, which Lua doesn't know about; if the high bit is set, here be dragons!


PREFIX: "MSP" ("MarySue Protocol")

SEP: "\1" - used to separate fields in a message. A single 1-byte (CTRL-A). Up to 32 fields can be sent per message, but in practice there should never be that many...

- When we receive an MSP prefix message, formatted in AceComm3 style, parse it into fields by f1, f2, f3, ..., f32 = strsplit(\1).

- Note, there may be only one field in the message, in which case, there aren't any \1's separating them. That's fine.



Version
Each field has a "version" number associated with it. Version numbers are positive integer numbers. Uninitialised fields have a version number of 0. Version numbers start at 1 when you first add the data to it. They increment every time the field changes.

If you've changed at least one of the V* or T* fields, you should also increment the version number for TT. You don't need to do that more than once if you're making several changes in a batch, but if at least one of the V* or T* fields has changed, you should increment the TT version.

If you keep a cache or memoisation (and you ought to, in some form) then you should keep track of these for the ones you've cached, so you can get a quick reply if they haven't changed since you last heard. If you don't have any prior knowledge or cached data for a given field, request version number 0 and you're guaranteed to get the latest one (if it exists).





(Please note, given the name of the protocol, and my sense of humour, I am deliberately taking the mickey in some of the Examples below. Please take all examples with a pinch of salt. These are not suggestions, although they should all be correctly handled by a compliant RP addon. If you actually call your character Crystal Dragon Princess Moonbeam Marie-Elisabeth-Rochelle LeNoir, and she's the devastatingly beautiful daughter of Thrall and Lady Jaina Proudmoore, she's a vampire slayer despite also somehow being a vampire, and she appears outwardly happy—but her eyes are full of pain—then I pity you. And your character. And your realm. <grin>)

For each fragment of an MSP message received, parse as below:

[Etarna]

REQUEST_FIELD:
  "?XX1" where XX is a two-character FIELD_NAME as defined below, and the number (which can be any positive integer number of one or more digits) is a version number field.

  On receiving this, if you don't support versions, or if the version number is < your current one for that field, reply with a UPDATE_FIELD reply for that field. If the version number in the request is >= your current one for that field, reply with a NO_NEW_FIELD reply for that field.

  Note: A version number in this field of 0 (which is guaranteed to be less than your version number, which should probably start at 1 and go up) indicates that the client wants this information unconditionally.

  Note: You can request the TT (tooltip) field as a shorthand for requesting VP, VM, VV, NP, NF, NM, NS and NT. That has its own version number, which increments every time one of the fields it returns changes; thus, requesting ?TT0 will get you the tooltip for any user. DE (description data) isn't part of this, but you can of course have more than one request a message...

If you don't understand/support the field name, you may either not reply, or send a !XX0 reply for that field (data hasn't changed i.e. it's still nil, and version number is 0). However, not replying should be safe, and is obviously easier.


NO_NEW_FIELD:
  "!XX1" where XX is a two-character FIELD_NAME as defined below, and the number (which can be any positive integer number of one or more digits) is a version number field.

  This is a confirmation that this field's current version number is <= to the one you requested (and is the one returned). This implies that the data in the field hasn't changed since the version you asked for, so you can re-use it. Upon receiving one of these, you probably shouldn't request that field again for a while.


UPDATE_FIELD:
  "XX1=fieldcontents" where XX is a  two-character FIELD_NAME as defined below, the number (which can be any positive integer number of one or more digits) is a version number field, and fieldcontents are the contents of the field.

Notes:

• fieldcontents may be empty (in which case, the = will be the last character in that particular message fragment).
• fieldcontents may contain any character except \1, including the = character. You may wish to use strsplit with a pieces parameter of 2 on the = to separate out the field name/version number and the payload.

[Etarna]

FIELD_NAME:

The following fields are currently defined as standard fields, but any two letters are allowed for a field name. Field names are case-sensitive.

You MUST support VP, VM and VV. You SHOULD support TT. Everything else is 100% optional.

Version Metadata
VP: Max protocol version you support. Currently, we're at version 1 here. Example: "1".
VM: The name of your RP addon. Example: "MarySue", "MyRolePlay", "ImmersionRP".
VV: The version of your RP addon. Obviously, addon-specific, and may have no specific meaning. Example: "3.1.52", "1.0b-7-g78f6272".

Names
NP: Prefix to your name. Examples: "Mr.", "Mrs.", "Miss", "Mistress", "Sir", "Lord", "Baron", "Overlord", "Crystal Dragon Princess".
NF: First name. (Note: This might not be your character's in-game name, and can be used to change that or even remove that, in which case supporting addons probably ought to display the OOC character name somewhere in the tooltip as well; doubly so if all the fields are blank.) If undefined, addons should assume that using the characters UnitName is appropriate here. Examples: "Moonbeam", "Elyissa", "Dulcamara", "" (i.e. no first name known), "?" (presumably suggesting the character's identity is mysteriously unknown), "Bob".
NM: Middle names. There may be zero, one, or more, separated with spaces or occasionally hyphens, one would assume. Examples: "Charles John", "Rebecca", "Marie-Elisabeth-Rochelle"...
NS: Suffix, i.e. last name. Appears after the middle names when written. May be more than one word. May be blank, and defaults to blank. May also be your character's UnitName, with the first name changed, i.e. Knotbrooke is actually "Mr. Knotbrooke". Examples: "Bobson", "Menethil", ", son of Durotan", "", "LeNoir"
NT: Title. Just as implemented in RSP of yore. Something that by rights would normally appear below your name. Examples: "Vampire Slayer", "Experimental Researcher".


To display a name we probably want something like this:—

Code: [Select]

   local displayname = ("%s %s %s %s"):format(NP, NF or UnitName(unit), NM, NS)
    displayname = string.gsub(displayname, "%s%s+", " ")
    displayname = string.gsub(displayname, "%s+,%s+", ", ")
    print(displayname)

Note, removing spaces before commas (to allow suffixes with commas to work cleanly) and removing runs of more than one space (because one or more field may well be empty). Note also, as a reminder, the empty string for NF is not nil, and so will override UnitName(unit).


Note: All the fields above this, when changed, should also increment the version number for the TT field below.

Tooltip
TT: Tooltip. A meta-field used as a short-cut to requesting all the fields commonly used in a tooltip. Reserved, and is blank on its own, but its version number is used. If this is requested, return the following fields (if supported/populated), in this order: TT, VP, VM, VV, NP, NF, NM, NS, NT (i.e. all the ones above this one, in this list of field names). You should increment the version number for this field every time you change one of the others this returns.

Efficient clients may wish to precalculate their reply to this, and keep it around for very quick sending, as this is likely to be a common request.

[Etarna]

Physical attributes
AG: Age, in years. Normally a positive integer. Please do not include the units, i.e. the " years" part. Probably (but not necessarily) a number, although "?" may be appropriate if it's a particularly rude question. Highly race-dependent, of course. Example: "10026", "16", "∞".
AE: Eyes. Short (one or two word, if possible) description of the appearance/colour of the eyes. Race-dependent, of course. Example: "Blue", "Green", "Sparkling Cyan", "Enchanting Glowing Golden", "Full of pain".
AH: Height. For the love of sanity, please make it simply a positive non-zero integer, in centimetres. That way, clients can do unit changes if they prefer feet/inches, or even do sliders. Highly race-dependent, naturally. Example: "183", "78", "202".
AW: Weight. For the love of sanity, please make it simply a positive integer, in kilograms. That way, clients can do unit changes if they prefer stones/pounds, or even do sliders. Highly race-dependent, naturally. Example: "0" (weightless, ethereal?), "79", "50".

Emotion
EM: Emotion. A very, very short description of the character's current emotion, as visible at a glance. Examples: "Happy", "Sad", "Outwardly happy".

Motto
MO: Motto. If the character has a motto, or a single line that sums them up, so to speak, feel free to put it here. Optional, of course. Should probably be displayed with “double quotes”. Examples: "Not all who wander are lost.".

Description
DE: Description. A textual description of the character's appearance. Think text adventure, if you typed > examine target — what would you read? This is the traditional RSP "description" field. Note: It's very bad form to put life histories here, or how you react to the character. Note 2: This tends to be a long field, in particular, there are no length limits, it's left to players' sanity to determine this... (Bad) Examples: "Devastatingly beautiful, her smile charms small animals and makes your heart flutter as you walk past her, even if you're a girl. Her subtly-pointed ears and slight fangs make her seem mysterious, and alluring. You should probably talk to her."   (Better) Example: "An unkempt, rough-looking night elf woman with a perpetual feral scowl on her face."

[Etarna]

History
Remember, some addons may choose not to implement any of these fields. These fields, in particular, should be used to aid and augment roleplay—not replace it. Your character is likely not aware of this information on first meeting the character, and it may conflict with what they are told...

HI: History. A write-up about the character's history. If you really must put your character's life story, do it here, and not in DE. Could be (very) long, depending on the roleplayer. Restraint is suggested.
HH: Home. The character's current home, wherever they consider that to be. Short, please. Examples: "Stormwind", "Dalaran".
HB: Birthplace. Where the character was born, if they know. Short, please. Examples: "Astranaar", "Argus", "Lordaeron".

Flags
FR: Roleplaying style. One of:—

• "" = (unknown)
• "0" = "None" (obviously, discouraged! and not the same as unknown)
• "1" = "Beginner" (still learning)
• "2" = "Casual" (i.e. some of the time)
• "3" = "Full-time" (i.e. all of the time)
• "4" = "Mature themes" (i.e. that mysterious setting we won't support by name, you know the one —may be displayed simple as 'Roleplayer' by some clients)

FC: Character flags. One of:—

• "" = (unknown)
• "0" = "Not roleplaying" (obviously, discouraged! and not the same as unknown)
• "1" = "Out Of Character" (OOC)
• "2" = "In Character" (IC)
• "3" = "Looking for Contact" (LFC)
• "4" = "Storyteller" (storyline-setter)

Others may be defined in future.

[Etarna]

——— reserved ———

[Etarna]

OK! Comments, anyone...?

[Moncader]

I should have read this first... but I'll just copy/paste it here since this place is better.

This doesn't just go to if the chat servers can handle it or not. This is also a user experience issue.
I don't know if things have change a huge amount in this area since I left WoW but I do know that if you recieve 10-20 whispers in less than a second you computer 'will' lag. I highly doubt this mod would work well in busy RP areas where at any one time there are 1-10 or more (if you are a storyteller or something, or just the main person of the event) people mousing over you at the same time. Even worse so, think about descriptions. Now there is no way to send your 2000 character long descriptions to everybody at once. This means that, due to the rules of chat throttling by WoW itself, you must send each 'piece' of your description to each and every person that wants your description. If there is one person that wants your description, it'll take about 2-3 seconds to get it to them. In the current system, everybody currently logged in will get that description, meaning nobody needs to ask for it again.
If we used whispers, now every single individual person that wants your description will need to directly query you for it. So if you get more than one person asking for it at a time... it won't take 2-3 seconds for both of them, it will take exactly double, 4-6 seconds. Three people? 6-9. And it will keepoint going up, possibly giving you a bottle neck (Meaning that if you get 10 people asking for some reason all of the sudden, by the time you finish those 10 requests, somebody definately will have asked for some more information from you, description or just regular. Also, you won't be able to request other peoples information while sending information to other people).

The other major issue that WILL slow any average computer down is if you are in a city, raid, or other crowded area. Worst case is that nobody has MRP. You MUST query each and every of the hundreds of people in real time as you mouse over them to get their information, or even just see if they are in MRP.
Even more so, there is a limit to how fast messages can travel. Often, when people are looking for MRP players, they go by their tooltip. If a MRP player exists, they have a different tooltip. However, in this whisper case, when you mouse over a person for the first time, you will only query them, and then it takes around .5-1 second (assuming they aren't already sending things to other people) to send their info back to you. THEN your tooltip canchange. This means that the average mouse over will not show you their information, defeating the purpose of the addon.

Well, I hope I have made my point
I bet this new addon won't work once it's user count grows above a couple hundrend per server.

[Etarna]

Points I've made before, but nevertheless, someone is actually trying it with MarySue, and pushing the solution to us. If it works well it'd clearly be an excellent solution for users because it doesn't eat chat channels and a better solution from Blizzard's point of view as they discourage use of the chat channels like this.

So, we may as well test the idea, and see how it actually behaves in practice.

In fact, both protocols scale O(n²)—but it's whether that broadcast happens on the server or the clients... yes this protocol may issue (very small) one-time queries to all players you mouse over. Hopefully, at a reasonable rate.

Currently, when you transmit your data, you transmit it once, and the server sends it to all players with MRP/RSP. You don't transmit it very often because of the load it causes the server. 1 transmit, n² receive. Same for each chunk of your description, and also each time you change it (this can be quite significant?).

So, that protocol scales according to the number of RPers on the server, clustering the load on the chat server.

With this, you transmit it to each one who specifically requests it who you are near. This means higher transmitting client load, but lower receiving client load because you're not receiving data from people who aren't near you. Unless, as you said, everyone's in Stormwind in a big RP event, which is why this needs testing (which I will actually do, yes). Lower server load if the total number of messages is lower. Certainly none of the messages in this will be wasted, and the only extra messages I can see are the extra ones sent to the server that would otherwise have been broadcasts (and much of the MRP traffic is actually client-client traffic sent to the entire channel, which is, I admit, a terrible solution—arguably if the clients can handle that, they can easily handle this).

You will, in fact, be able to request other people's information while sending information to other people. Putting a ?TT0 query (5 extra bytes) on the beginning of your reply to their query would fetch their tooltip data in return—and, ChatThrottleLib does prioritisation queues of its own (hierarchical token bucket by the look of it).

In terms of descriptions, people don't bring them up anywhere near as often as they mouse over, I think at least. Maybe broadcasting those is a false economy—I don't know. It's something to test rather than theorise over.

Certainly, from my tests, SendAddonMessage has a much lower overhead than normal whispers, and, it appears, a much different queue and throttle rate (because it doesn't have the anti-goldseller-spam mechanisms in place?).

In short, theoretically I'm right there with you, and it's nothing I haven't said before. This particular time, however, I'm not willing to leave it just to theory. I'd quite like to try it and see how it really shapes up in practice.

What's the busiest RP server you know of?

[Sec]

This is Avanina from Curse.com and Sec on WoWInterface.com. I'm the one who got in contact with many of the other "flag" RP addons.

Regarding Server Choice. If this has not already been decided.

On US Servers, I'd say the "busiest" two servers are Moon Guard and Wyrmrest Accord. Moon Guard used to suffer through a queue (and still may). Wyrmrest Accord is newer and has a lower population, but it seems many people have transferred to this server to seek out Roleplay.

I formerly played on Moon Guard but have been a member of Wyrmrest Accord since January or so. I've found the RP community on Wyrmrest Accord to be thriving. Peoples' efforts to keep the Roleplay community going have been strong efforts and the work is paying off. I play solely on Horde and so I cannot comment on Alliance, but the general brake down is: Alliance seems to have more "Tavern-RP" (random and casual) while Horde seems to have more "Event-RP". This is not to say Alliance does not make events or Horde doesn't have casual RP. Brill's tavern for example is stage to a guild that acts as staff (<Golden Goblet Inns>).

I'm no community guru but will give you a hand with testing if anything gets off the ground.

WyrmrestAccord.net is a player-run community site. Please check it out. Not sure if such a thing exists for Moon Guard.

About "Open Invitation"

I'm not sure if any other author has checked in on this forum in order to see and contribute to the ideas you have put forth here. I'll send some PMs out in order to bridge the potential communication gap. I know that at least one author who is not currently playing WoW or developing their addon (FlagRSP2 written by Alexia).

[Antiarc]

Howdy --

Sorry for the (extreme!) delay in responding to this. I never check my Curse messages, and just sort of fire-and-forgot MarySue (I'm not an active roleplayer, but wrote it at the behest of some guildmates), so it's just sort of been hanging out there.

I'd love to devise a unified protocol. When I was writing MarySue, I was actually thinking about just abstracting it into a generalized data exchange addon that you could use for most any purpose (mouse-over LFG, for example), but then I got lazy and just knocked it out. A strong protocol could definitely be of benefit to the community, though.

I'll read over the proposed protocol and give some feedback between pulls. I'm just about to start a raid, so I may get distracted, but if I do, just poke me on #wowace at irc.freenode.net, or via email at cheald at gmail. I get very distracted these days, so the chances I will get sidetracked are high. *grin*

I'd made the point on Curse that a whisper comms protocol is only worse if a person comes in contact with over half of the people that would otherwise be in a channel during a playsession, via my quick napkin math. It's worse in the worst case, but better in the average case, I believe, and it gets around the problem of a consumed channel. Additionally, I extremely heavily doubt that given the use of a good throttling lib (Like...ChatThrottleLib), that one could reach a point that the traffic transfer was dangerous or contributory to performance problems, and I say that as the guy who wrote Threat-2.0, which did so much traffic that Blizzard gave us threat information directly from the server in 3.0

While the big-O for whisper comms is worse than channel comms from the client's perspective, I think that the Blizzard infrastructure is more than robust enough to handle it, and the improvement in user experience is vast.

[Antiarc]

Okay, having read over the spec, I like it, with the exception of the flags. Those should be a bitmask, in my opinion. Hex bitmasks are extremely compact, and allow for a very extensible flags format that would potentially allow communication of fields where there might be multiple bits set in a flag set. This allows us to compress all the RP-related flags into a single bitmask, which a) consumes fewer fields, b) consumes less bandwidth, and c) lets us use bitmasks (I will use bitmasks at any opportunity!).

Since this is a pull protocol, we should probably define a minimum throttle time between requests to a given client. I think I have it set at 20 seconds, and it'll poll non-responsive clients every 5 minutes (in case they were to install the mod), looking over the code. I'm not sure what the practical use cases are (you mentioned "emotion" as one of the fields, which could be changing fairly regularly). However, I think we could do this:

1) Increase the standard throttle time to something like 5 minutes.
2) Don't re-poll non-responsive clients. It's an edge case, and it'll work next session, at the benefit of saving a lot of bandwidth in avoiding re-polling people regularly. If a client pings us, add them to the "valid clients" list, so they they become a valid poll target.
3) When I send out my info, I should keep a table of people I've sent it to, with the transmission time. When my information changes, I should do a push of those updated fields to those people (perhaps, for sanity's sake, we limit this to a queue of the last X people we interacted with) via whisper comms, just as though they had requested information from us. This gives relatively realtime updates without being overly spammy with the pull requests. The net effect should be that when people are changing things like emotion or whatnot over the course of an RP session, the people who they are RPing with will have that information updated in a timely manner without having to regularly query for it.

In addition to the AH and AW fields, which are numeric, I'd like to allow for some kind of string equivalents. Some people don't want to use numbers (especially weights), and it seems a bit constrictive to force it. I do like the idea of converting between units, but I'm not sure it's necessary - US players will more or less always use feet and inches, while EU players will more or less always use meters/centimeters. In this case, I think it might be best left up to the individual as is most appropriate for their locale.

[Sec]

I can't help but wonder... Is this seemingly wonderful idea stuck dead in the water?

Here is a thread I wrote on my RP server's community website - for the community, by the community.

http://wyrmrestaccord.net/content/marysue-rp-addon

The purpose of the thread was to compare MarySue to xtensionxtooltip2 addons in order to raise awareness of MarySue and what it is capable of. In the course of the thread, I mentioned to posters that if they experience bugs with their addon of choice, they ought to notify the addon authors. However, there are a minimum number of bugs reported - it seems the average roleplayer does not voice what's wrong. I feel very loud in such a quiet crowd.


I know that addons, when not maintained, often either fall to the wayside or are patched/forked by other individuals in time. I do not know what will happen to the handful of RP addons that the average roleplayer has to choose from, but I believe that Cataclysm will be an ideal opportunity to offer something improved.

I would much rather see the selection stay as varied as it is right now, because each addon has features that are unique.

Aside from MarySue as a new option, it seems that MRP has the most active development being done to it - if past attempts at conversing with RP addon authors are any indicator.

If MarySue and MRP were to be the only two addons working when Cataclysm released, I'd hope that MarySue and MRP were cross-compatible, to share descriptions.

If only MRP was working for Cataclysm, I'd hope the following:

- MRP were to use the MarySue Protocol to avoid problems like descriptions slurring with Blizzard's "drunk" text effect, delayed updates, memory usuage.
- MRP would have modular features such as:
      - Description browser - save descriptions after viewing for later inspection; perhaps via clicking a square labeled "S" in the description window (for example).
      - History field - some folks do not care to input or read other peoples' histories.
      - Compact description window that is along the design of MarySue; which is based on other addons, of course.
      - Toggle for the MRP circular button
      - Option menu integrated into Blizzard's Addons menu
      - Ability to disable showing descriptions when in combat or other conditions like in an instance, raid, or battleground.
      - An "RP Mode" that hides all UI elements aside from chat boxes. This can be emulated by another addon (Kong on wowinterface.com) but would require manually being set up by the individual. This feature currently is unique to IRP (Immersion Roleplay) as far as I know.


I do apologize if any of these features are already present within MRP. I have not used MRP for some time simply because I prefer the aesthetics and features of another addon. That really is the one reason why people prefer X over Y when it comes to RP addons; people are not choosing an addon for improved performance, but individual features or layout. The features listed are in some cases unique to one addon, though I am sure the list is not complete. If MRP were to have these features in a modular package, then individuals could pick and choose what they wanted, while still being able to interact with MRP users who have different modules enabled.



In conclusion,

If you (Elandru) still have interest on MarySue Protocol, please post a reply. I can send PMs to the various RP addon authors and state that an outlined protocol has been suggested. I hope that this may peak other authors' interests for improving their own products for the benefit of themselves (if they use their own addons!) and their userbase. If no authors show interest, I humbly ask that you consider what I have stated in this post. With Cataclysm on the horizon, and not knowing how the current RP addons may be affected, I would hope that you might be interested in improving and maintaining MyRolePlay.

A modular MyRolePlay with an improved method of data transfer is something that I think the roleplay community at large would appreciate. I do not know what will happen when the dust stirred by Cataclysm finally settles, but I would like to see at least one roleplay description addon standing - all the better if it works more effectively than before.

[Etarna]

I’m mainly worried about seeing if it explodes when implemented, because it scales O(n²).

On the other hand, so does the channel.

I don’t want to break anything, and given the very extensive change, and that we’d essentially be using ChatThrottleLib and almost nothing else, it’d be almost unrecognisable.

I do have a small pile of highly experimental source for this, and as we get closer to Cataclysm, I’ll be testing—MRP to make sure that it at least some minimum level of works, and the experimental MRP4 to see if it blows up horribly on a busy realm like Argent Dawn.

Maybe.

[Sec]

Certainly understandable so far as the possibility of it blowing up. Though, I have good faith in ChatThrottleLib, as it has been used in the past for heavy data transfer - it will be something interesting to see, for sure. If ChatThrottleLib is used, it should save some coding, right? That seems like a good thing to me.

If you do release anything for testing that utilizes this MarySue Protocol, would you be making such beta/test copy public?

Both North American and European server lists have an Argent Dawn, and both are Roleplay servers. I have a North American copy of the game, and would be happy to test it on my home server (Wyrmrest Accord) as I'd be unable to play on EU's AD. There may be some folks interested in testing on my server; many people liked the idea of MarySue (smaller, more efficient), but did not want to switch because it lacked cross-compatibility. We are sitting at Medium/High server populations, differing on the time of day, and most folks are roleplayers - some RP server have more non-roleplayers than actual roleplayers.

I must admit, I am also a bit curious about MRP4, to see how it differs from the current MRP if at all.

You're the only RP addon author (amonst the xtension addons) who has made any mention of active development in relation to Cataclysm. I suppose it's just a matter of wait and see what the other authors do, if anything.