TTR logo

The Text Reformatter User Guide


Overview

    TTR is designed to reformat text, more specifically fanfiction text. It can handle both propose or script style. TTR helps reading, printing and publishing story. TTR analyses the text and tries to guess the structure of the text. TTR can unwrap text (putting all the lines of a paragraph back together), or wrap text (splitting a  paragraph into many lines). The reformatted text can then be exported as text, or as HTML, to a file or the clipboard.

    Why put paragraph back together? For printing a text, your favorite word processor can handle things very well: printing on two columns, justifying the text, reduce font size... Yet, do to this, the text need to be organized as paragraph. This is what TTR do.

    The main usages of TTR are:

    I already can here you saying: "Are you crazy, how can you print Neon Exodus Evangelion ?" Well, this little story which is about 2Mo long can be printed using only 236 pages... This could not be done if paragraphs were not put back together.
 

Quick Start

Quick concept

To reformat the text, TTR uses a reformatting engine. The engine applies a set of rules to determine if a given line begins a new paragraph or not. There are many rules, which can be turned on or off, each suited for different kind of text format. In the following sections, we'll study the most typical text formats and the matching engine settings.

In the following sections, I will use excerpt from many fanfictions to illustrate the examples.

Quick Usage

First you should get a look a the short " Understanding the view " section to get to know the user interface.

Now, you may use TTR in many different way. What is presented is the way I use it, which I found is quite time saving.

Makes a copy of the text(s) to reformat in a given directory (TTR will overwrite the original text on saving). Then drag and drop the copied files from the directory which contains the version to reformat into the main window of TTR.

In the following section, I will drive you through the reformatting and the most typical text format you encounter in fanfiction texts. All those example texts are available in the examples directory, in the same directory where you installed TTR. Before you start, makes a copy of those examples into another directories so you can save the result of the reformatting process.

Reformatting "well" formatted text

That one of the most common kinds of text now days. We'll see how to reformat this so you can put it in a word processor or view them with your HTML browser.

Well, there are basically two kinds of "well" formatted text. The first one is illustrated by the following example:
 
With a scream and a leap Sailor Sun launched herself to where 
Uranus was guarding Saturn and Neptune. Sticking her arms in the 
air, tried to ignore the way she could feel the bones grate in her left 
arm. "Sailors, join on me!"

Despite not knowing what she planned, all the Sailors moved. 
Teamwork was the key to their success. If Sailor Sun knew something 
they did not, they did not have time to question her. Sun took one last 
look around her, making sure her friends were close enough. Closing 
her eyes, she called on the one other attack she knew she had. 
"SOLAR FLARE!"

A sudden concussion knocked most of the Senshi off their feet, and 
shook the youma, causing them to pause. Even though Moon fell 
straight onto Sailor Sun, the red haired girl did not move in the 
slightest. While she channelled massive quantities of power, the 
ground began to shake, and an ominous thunder rose from all around. 
The youma had just rejoined the attack when the full force appeared.

Centred around Sailor Sun was a small area of calm in which the 
Senshi and  two youma hid, too stunned to speak or move while the 
force of nature raged beside them. A third youma almost joined them, 
but it had been caught on the invisible dividing line between safety and 
destruction. With half it's body vaporised in a raging inferno, the 
youma fell to the ground and began to dissolve. All the Senshi could 
see was a dome of fire, covering them, surrounding them. A muted 
roar of flames could be heard, as though far away. Inside the dome, it 
was calm and cool. After seeing what had happened to the youma 
caught in the surrounding fires, none felt the desire to try putting a 
hand outside.

-- Excerpt from "Sailor Ranko", by Fire.

Now, look at the text above and tell me what are the clues that indicate you where a paragraph begins. Ah, that one is easy: each paragraph is preceded by a blank line. That it, you got it! The reformatting engine just does that. You adjusting the engine setting so that you activate the rule:
    "when a line is preceded by a blank line, that line begins a paragraph".

Now, let's study the second case:
 
       Nene, stretched and yawned, and turned back to her computer.  It was
late, after   midnight. Naoko was already gone, and it was probably time for
her to head out herself. It   had actually been a slow day, and she had a
normal day off tomorrow. No Knight Saber   business either, yay! It should
be heaven. A sigh escaped her lips. Even work was better. It   was nice that
the ADP had finally got to computers on the desks of the office workers.
With some of the trips that she had installed, she could get some flags to
pop up, and she   wouldn't have to leave her desk to get some of the more
basic information for the Knight   Sabers.
       She had even put in some alarms about her, in case somebody dug in
too deep, and   figured her out. It was something that she put in after Dr.
Mirriam's attack on the ADP,   when a girl named Lisa figured out that she
was a Knight Saber. That was too close, Nene   thought. Ah, well, grab my
jacket, and I'm out of here. Just as her hand reached her coat,   her
computer beeped, in a seemingly random fashion. "Uh, oh." Nene said as she
stared at   her computer across the room.
       That was probably the biggest understatement ever uttered by Nene.
Sliding   around to look at the screen, she saw her future at the ADP
disintegrate. "Oh, no!" she   moaned as she read a memo being sent to the
front desk concerning her arrest as a mole   for the Knight Sabers. Nene
flinched when she saw the part of the memo that warns that   she is probably
armed and dangerous.  Shoot to capture. A quick glance revealed the
officer in charge of her capture. Matio Dansk, a known Genom mole. "I'd
better get out of   here. Dansk is probably already on his way over!"
       With a flurry of keystrokes, she sent out a handful of specialized
commands that   would cripple the ADP for about an hour. And more. If she
couldn't get out in that time,   she wasn't getting out at all. The last
command sent a message to the Knight Sabers about   her cover being blown.
And with that, she fled out the door to the stairs, now invisible to   the
cameras and sensors.

-- Excerpt from "Maximum Polarity", by Arthur Hansen.

Well, this time the rule would be something like:
    "the line is indented (it has space at the beginning) therefore it begins a paragraph".

Let's reformat those two texts...

First, open your copy of the example file s_ranko.txt which can be found in the examples directory where you installed TTR. You can do this using the "File/Open..." menu, or by dragging the file from your file manager (Explorer, Windows Commander...) and dropping it into TTR' s main window.

Open the settings dialog (select the menu in the "general settings" menu or press the button in the toolbar).

You should see something like that:
Analysis settings dialog, showing the Block analysis modes Tab.

The options in the top frame box apply to rule that determine if a line begin a new paragraph. The options in the middle frame box apply to rule that prevent a line from beginning a new paragraph (those rules override those of the top frame box). The options in the bottom frame box determine what character may compose a separator line. Separator are usually use to mark time or space discontinuity between two scene ("*    *    *" for example). 

As we can see, the rules that are active by default are those we need to reformat the to previous example. Close the dialog and press the Tab key, that will reformat the selected block using the analysis engine settings. Since no block is selected, the action is applied to all the text.

You should now see the following:
Screen shot of TTR showing the refomatted text.

As you read in the section "Understanding the view ", a long red right arrow indicates the beginning of a paragraph, and a short violet left arrow indicates that the line belongs to the same paragraph as the previous line. Now, scroll down to the end of the text using the page down key to check that the analysis tool did not make any mistake.

Everything should be ok, so now we'll save the text as a text file and an HTML file. That is simple, press Ctrl+S to save it as a text file (that will overwrite the original text), and press Ctrl+H to save a copy of the text in HTML. The HTML file will be at the save location as the original text but with the .html extension.

Open the HTML file in you web browser. You can see that you get a clean HTML file that can resize the paragraph as you change the window size. And it that was done in a matter of seconds !!! Click here to see an example of the HTML output you get.

Open the text in your preferred word processor, you can now change the presentation. I personally like to justify the text, use a 8 points font, 2 columns of text, a separator line between the column. I found that those setting easier to the eyes (they move less, so you tired as much as you would otherwise). If you don't have any word processor, just launch WordPad, open the text, and in the Display/Options menu, change the line break setting so it can resize with the window...

You can reformat the text of the second example by following exactly the same steps. The example file is maximum-polarity.txt.

Reformatting text without format clues

Let's get on a harder example:
 
Running up the stairs to the temple's courtyard, they could hear
the demon's coarse breath just behind them. As they reached the top
of the stairs, under the archway, Yohko caught her toe on the edge
of a flag stone and went sprawling. Because of her training, she
managed to twist in mid air to land painfully on her back, soul-
sword at the ready for one final desperate defence. To her
amazement, instead of the demon dropping down upon her, it stopped
dead in mid leap and was thrown back down the stairs by some
unknown force. It then proceeded to stalk back and forth at the
base on the stairs, thwarted, but not about to give up.
Leaning on her soul-spear, and looking about to collapse in spite
of it, Azusa panted heavily. "I don't get it, Mistress, what's
holding it back?"
"I think its because this is a holy site. The powers that this old
temple is dedicated to must still hold sway." Yohko mused.
"_Old_ Temple, Yohko-sama? This place looks almost new."
"Yeah, its been abandoned since I was a little girl....Huh?"
Looking around, Yohko noticed that the temple, and the small house
that was off to one side of the courtyard, was indeed looking in
very good repair. Yet Yohko remembered coming by here two years ago
on a shopping trip with her grandmother and it had looked about to
fall in on itself. Just then a sliding door on the veranda of the
house opened, and a tall, well dressed woman came out, a mug of tea
in her hands. As she sipped from the mug, her eyes roved the
grounds with a bored expression in them. When they got to the pair,
standing inside the gate leaning on their weapons, she straightened
with a look of interest. "Belldandy! Could you come out here for a
minute?" she called over her shoulder, her voice rich and sultry.
As she proceeded to cross over to them the slight breeze wafted her
long white hair to one side. The devil hunters could see that she
had strange tattoos on her forehead and cheeks.
When she was near Yohko bowed to her. "I apologize for the
intrusion ma'am, but we are Devil Hunters you see and..."
"..And you think you're gonna find some *here*?" The woman said, a
hint of amusement in her voice.
"No, uh, actually we were running from that one...." Azusa said
pointing down the stairs.

-- Excerpt from "Oh My Devil-Hunter", by Mark Babcock.

Well this one is harder. There is neither indent nor blank line to indicates paragraph. The best clue we have is that the previous line ended a sentence, and therefore possibly a paragraph. Another one is that the sentence ended nowhere near the end of the line. So we have a rule like:
    "The previous line ended a sentence".

The second clue (the line was short) is not used yet (see planned features ). The current formatting is hard to read. But using TTR you can refomat the text so that paragraph will be indented making it much easier to read.

Because this rule is not 100% sure you should skim through the text to check the result. For example, the sentence in red would have been detected as ending a paragraph which it does not (though those errors usually don't matter much). See the detail section on how to use some rules of the engine to improve the result.

Open the file OhMyDevi.txt. Open the analysis setting dialog and go in the second Tab. This Tab is used to deal with text of that kind. It is the "End of Sentence Mode". Check the "End Of sentence mode activated" option to activate the EOS mode. The edit field allows you to specify which characters may end a sentence. You should have something like this:

Analysis settings dialog, showing the End of Sentence Mode Tab.

Now, close the dialog by clicking the Ok button, and press Tab to reformat the text. But remember in the example, there was a line that should not have been indented. Search that line (press Ctrl+F and type sultry). Glue the next line by selecting it and pressing the left arrow . You can now save the text. Load it in your browser, and you get a nicely presented resizeable text. Click here to the reformatted HTML text. Well, you now the basic now. Let's see one last example where I quickly introduce a few more analysis settings.

Reformatting script text

Sato: [rubbing head] Gomen, Yamakawa-san, but we don't have time for the long
      version. [looks at Ukyou] The Moon-Child will bring both happiness and
      sadness.
Ukyou: Is that why you were so hesitant to take the Moon-Child?
Sato: [sighs] Part of it is to avoid disturbing the natural harmony of things.
      The other part is... I wasn't sure I could take care of her.
Ukyou: [shock on her face] You would have left a BABY alone?
Sato: That is not a mere baby, that's a...
Ukyou: A Moon-Child! Chikusho! I won't let you just abandon her!
Sato: But, Ukyou, that's not what I meant to...
[Ukyou gets up, cradling the Moon-Child in one arm, and quickly opens the
door.]
Sato: Wait...
[Ukyou pulls down an eyelid with one finger, and sticks her tongue out, then
stomps through the doorway and slams the door behind her.]
Sato: [rubbing head with hand] All I wanted to say was that I could've gotten
      Furuya-san to help me. What was all that about?

--Excerpt from "Daigakusei no Ukyou: As You Like It", By David Tai & co...
Notes that for this example I modified the format of the original text by removing all the blank line between the paragraphs (that would be way too easy ;-) ). You can found example in the file named daigakusei-no-ukyou.txt in the example directory.

Well, that format is nasty but it's not that bad. The first rule we can find is:
    "a line that contains ':' (after a character name) begins a paragraph".

The second one would be:
    "a line that begin with '[' begins a paragraph".

Set the settings to activate those rules. The settings dialog should be as follow:
Only the indentation rules "line is a blank line", "line contains ':'" and "line begins with..." are enabled.

Press the Tab key to reformat and the magic is done. Child game isn't it ?


User Interface

Understanding the view (all those colored arrows, what do they means?)

Here is a screen shot the view while the text is being reformatted:
Screen shot of TTR showing a selected block and a reformated text.

What does those arrows indicates ?

The button is used to change the analysis settings (open the settings dialog).

The button is used to preview the reformatted output as HTML. A temporary HTML file is created and automatically open in your web browser.

The first group of 3 buttons in the toolbar ( ) is used to manipulate block. You can see that because there are more than one "operation marks" (arrows, line).

The second button group ( ) is used to manipulate line one at a time.

In the third button group, is used to reformat the current block, or all the text if no block is selected. is used to guess what is the right margin to use with the right margin rule .

This second screen shot illustrate the e-mail and news mode:
Screen shot of TTR view where the '> > ' text used to mark sendee part of the message are shown in orange.

The dark orange highlight in the left margin ('>' hightlighted with dark orange ) indicates the quotation mark used to identify the quotation depth.

Opening a text

There is three ways to open a text:

Moving around in the text

The current line (where the "cursor" is) is showed in blue. You can change the current line by clicking with the left button (be careful not to move the mouse to much). You can also use the following key commands:
 
up arrow Move up by 1 line
down arrow Move down by 1 line
page up Move up by 1 page
page down Move down by 1 page
home Move to the beginning of the document
end Move to the last line of the document
Ctrl+F Search text
F3 Search the next occurrences of the searched text.

Selecting a block

You can select a block of text using 3 different ways: The select block will appears in a different colors and markers will appears in the left margin indicating the beginning and the end of the current selected block.

Reformatting the text

There is two ways to reformat a text: manually and using the analysis tool. I will first present all the manual way to reformat a block and then will see how to use the analysis tool.

Changing a line state (glued, original, indented)

Changing a block state (glued, original, indented)

Shift+Tab Indent the block
Ctrl+Tab Glue the block
Tab Run the analysis on the block
Spacebar Reset block to its original indentation

Reformatting with the analysis tool

Saving a text

Keyboard shortcuts

The following table list (hopefully) all keyboard shortcuts:
Opening a new text
Ctrl+O
Open from text file
Ctrl+N
Paste text from clipboard
Previewing the reformatted text
Ctrl+P
Preview the reformatted text as HTML
Saving the reformatted text
Ctrl+S
Save as text
Ctrl+Shift+S
Save as text and ask the filename
Ctrl+W
Save as wrapped text
Ctrl+Shift+W
Save as wrapped text and ask the filename
Ctrl+H
Save as HTML
Ctrl+Shift+S
Save as HTML and ask the filename
Ctrl+C
Save as text to clipboard
Ctrl+X
Save as wrapped text to clipboard
Ctrl+L
Save as HTML to clipboard
Block actions
Tab
Run the reformatting engine on the selected block (or all text if no block is selected)
Shift+Tab
Makes each line of the selected block a distinct paragraph
Ctrl+Tab
Makes all lines of the selected block part of the paragraph starting immediately before the first line of the block
Space
Makes each lines of the selected block a distinct paragraph, but keep the original indentation when saving
Ctrl+R
Set the right margin used by the right margin rule to the length of longest line of the selected block
Line actions
Left arrow
Makes the current line part of the same paragraph as the previous line
Right arrow
Makes the current line a distinct paragraph. Press twice to toggle from glued to indented
Moving into the view
Up arrow
Moves the current line up. Scroll the view to keep the current line centered
Down arrow
Moves the current line down. Scroll the view to keep the current line centered
Page up
Moves the current line up by one page. Scroll the view to keep the current line centered
Page down
Moves the current down by one page. Scroll the view to keep the current line centered
Ctrl+Left arrow
Scroll the view to the left
Ctrl+Right arrow
Scroll the view to the right
Selection
Shift+Up arrow
Makes a new selection, or extends the current selected block one line up
Shift+Down arrow
Makes a new selection, or extends the current selected block one line down



How to...

Read an unwrapped text from my web browser without scroll all the way to the left

God know what a bother it is to stumble upon such text. This is where you use the new clipboard facilities. Here are the steps:

Quickly reformat a multiple parts story

Here are the steps:

Settings

Analysis options

Those are the options found on the following tab:
Settings dialog, analysis options tab

Indentation rules

Indentation rules
Line is a blank line
when a line is a blank line, that line begins a paragraph.
Line is indented
when a line is indented (it has space(s) at the beginning) then that line begins a paragraph.
Previous line is a blank
when a line is preceded by a blank line, that line begins a paragraph
Line contains ':'
when a line contains the character ':' (a colon), that line begins a paragraph.
Line begin with...
when a line begin with a character specified in the edit field beside the option (bracket and square bracket are the default), that line begins a paragraph .
Left margin...
specify the number of 'space' that make up the left margin of the text. If activated, only line with more that the specified number of space will be considered indented.
E-mail and new rules
when activated, e-mail and news quotation marks (the "> " string at the beginning of quoted e-mail lines) will be virtually stripped before the text is analyzed. If the quotation mark of the current line is different from the quotation mark of the previous line, then the line begins a paragraph, otherwise the other rules are applied as if the quotation marks did not exist.

Rules preventing a new paragraph from beginning

Rules preventing an indentation
Enclosure...
prevents a paragraph from beginning inside an enclosed part of text (a paragraph can not begin inside a pair of square brackets for example).
Line does not begin with an upper case letter
prevents a line from beginning a paragraph if it does not start with an upper case letter.

Centering rules

Centering rules
Separator line...
if a line contains only the characters specified in the edit field beside the option, then that line is centered.

End of sentence mode

The end of sentence mode analysis options are specified on the following tab:
Settings dialog, end of sentence mode tab

End of sentence mode activated
use that options when a text structure is not apparent (there is no blank or between paragraph, and paragraph are not indented). When enabled, a new paragraph will be started if the previous line ends a sentence (ends with one of the character specified in the "End of sentence characters' edit field).
Right margin
the case of use of this options is similar to the end of sentence mode. Yet, you should be more careful when using it. When enabled, a new paragraph will be started if the first word of the current line, when added to the previous line, produce a line longer than the "right margin" length specified in the edit field beside the option. This options can produce good result on text that is well formatted. You usually use the "determine right margin" action to find out what is the actual maximum line length in the file (toolbar button , or Ctrl+R). When the right margin mode is active, the text is displayed using a fixed width font and the right margin position is shown with a vertical red line).
End of sentence characters
specify the list of characters that may end a sentence.

You might want to enabled the "Rules preventing a new paragraph from beginning " to get better result when using those options (especially the right margin rule).

Output format options (when saving)

This are the options use to customize the output format. They are found on the following tab:
Settings dialog, saving mode tab

Jump one line after each paragraph insert one blank line between each paragraph if enabled.
Remove ending hyphen
if a line ends with '-' then it is collated with the next line and the hyphen is removed
Indentation format
used to define how the indentation of indented paragraph (those starting with a red arrow pointing to the right) is outputted
Use Tabulation
tabulation(s) is/are outputted to indent the paragraph (I recommend this if printing, when loading the file in your word processor, you can set up the actual tabulation length)
Use spaces
space(s) is/are outputted to indent the paragraph (I recommend this when publishing on the web in text format)
Keep original
output the the original 'indent' string.
Number of spaces or tabulations per indentation
specify the number of spaces or tabulation that is outputted for an indented paragraph.
Wrapping at column #
specify the maximum number of character per line when wrapping the outputted text

When should I use that option?

For prose text, you will usually use the following rules:     If those rules are not enough (paragraph are not indented and there is no blank between paragraphs for example), then it's time to get the heavy weapon:
    Prefer "right margin" to "end of sentence mode" when facing a text with a lot of  poetry or quotation that would mess the "end of sentence mode". On the other hand, "end of sentence mode" is easier to use and usually provides a good output quality.

    You usually combines those rules with:
For script text, you will usually apply the following rules:
    You might also want to use:
    Those rules usually provides better reformatting quality.


Planned features

Known bugs

If you found some other bugs that are not listed here, then contact me .

Things to come

If you have any other idea, feel free to send it to me .

Where Can I Get TTR?

You can get the latest available binary and source release of TTR at http://gaiacrtn.free.fr/index.html .

Author

The Text Reformatter was written by Baptiste Lepilleur.

The first version of TTR used to run under DOS, about 8 years ago. It was later ported to Windows when I started learning MFC (and started running on all the 16 bits bugs of that supposed 32 bits OS...). Nowadays, It is kind of a laboratory for me.

Contact

You can send suggestions, grammar and syntax errors to Baptiste Lepilleur at gaiacrtn@free.fr . Any comments is welcome.

License

This software is covered by the GNU public license. Click here to see the text of the license. 
Copyright ©2000-2002, Baptiste Lepilleur . http://gaiacrtn.free.fr/index.html