  	-----=== HF ===-----
	
H F  -  H O W T O 

LINUX HAM RADIO SHORTWAVE PROGRAM
FOR RTTY, AMTOR, GTOR, PACTOR 1, MT63
WITH INTERFACE TO F6FBB MAILBOX 

Version 0.7 JANUARY 2005


1) Introduction :
   ==============
   
The hf program suite implements the amateur radio protocols
RTTY, AMTOR (SITOR), GTOR, Pactor-1, and MT63. 
It uses a normal soundcard as modem. 
The speed change in Pactor (100 to 200 baud) is done automatic. 
A PTT (push to talk) signal can be output via RTS pin of a serial
interface.
   
The implementation of the protocols uses a binary called hfkernel,
which runs as daemon in the background. 
Hfkernel provides an UNIX domain
socket to communicate with the user interface andd mailbox interface -
like the terminal program hfterm. 
UNIX domain sockets were chosen since they could be easily changed to
internet domain sockets in the future, to let the hfkernel process run
on a different machine than the user interface.

Hfkernel internally uses three threads: 
one is the modem code which reads/writes to the sound card in real-time mode, 
one is the running protocol, and one that handles socket I/O.

Hfkernel has been made for the OSS sound drivers, but it also runs with
the new ALSA drivers, because they have an OSS emulation that is being
improved steadily. But it does not run with every soundcard, because it
uses the mmap() system call (for low latencies, needed for real time
mode in Pactor and Amtor ARQ), which is not supported by some cards.
Crashes are possible. 
The experimental option '-n' (no mmap) is working good very often and 
is now also available for the 2 of the 3 calibration programs in the package,
that use mmap().

For the ARQ modes and MT63 a calibration will be necessary in most cases.
Three calibration tools are included.

The terminal program hfterm is the graphic user interface to hfkernel.

The start script hf reads and tests the configuration file and starts 
hfkernel and hfterm.

hfterm can connect via a TCP port with the well-known mailbox program
F6FBB (and, as we hope, with any other mailbox program) 
to be used as an automatic Pactor, Amtor or MT63 mailbox.

The program was made in 1997 by Tom Sailer (Thank you!) and is being 
developed by Axel Krause, Gnther Montag and many friends.
Since V.0.6.6 Pawel Jalocha's MT63 is integrated.

Please contact us by the mailing list!


2) Installation, Configuration, first test:
   ========================================

Requirements:
Hardware:
Pentium from 133 MHz on. The soundcard must support mmap().
Software:
hf makes use of gtk+ toolkit (Version > 1.2.0), which is in many
linux distributions. To know whether this is installed, type 
'gtk-config --version'.

hf-<version>.tar.gz:
unpack:
# tar -zxvf hf-<version>.tar.gz,
go into directory:
# cd hf-<version>
start configure script:
# ./configure
(options are visible with configure --help, but 
normally you need no options)
compile it:
# make
install it, as you do not have write permission to /usr/bin, as root:
# make install

'hf-<version>.rpm':
install: 
# rpm -i hf-<version>.rpm
or, if false error "gtk missing":
# rpm -i --nodeps hf-<version>.rpm

The global configuration file is /etc/hf.conf, normally you only
have to enter your serial line for PTT here. 

First test:
On a console:
Start once as root, once as user
# /usr/bin/hf
without options.
The start script hf tests the configuration and gives hints.
If started as user, it makes a directory "hf" in your home directory
and fills it with example files.
(If you do not use the hf script, do this manually, the
examples are in /usr/share/hf/hf-examplefiles).
If successfull, after the first messages a row of numbers 
rolls over the screen:
corrout .... intermediate ...
This means, the "heart" hfkernel beats regularly.

If this runs, then under X:
Open a command window whith 'ALT + F2' and enter 
# /usr/bin/hf
Hfkernel will appear in an xterm, and then the graphic hfterm window
will appear in the foreground. 
To test signal input and the soundcard gain, open the spectrum display 
with F2 or <Shft>F or the Spectrum button.
Decoded characters should appear in the rx window.

If hf runs, create a link onto the desktop with 
ln -s /usr/bin/hf /home/your_user_name/Desktop/hf'. You also
can right-click on the desktop,  'new', 'application' ... but as an old
linux rabbit you will know that. So You can start and end the program
whith one mouse click. 

As said before: Your personal directory (configuration, fixtexts, Log...) 
is "hf" in your home directory. It is made at first start of hf and filled with
example text macros. 
Your personal configuration (call, name ...) is done menu-guided from hfterm.

The documentation is in /usr/share/hf, and - depends on distribution - 
also in /usr/share/doc/hf or in /usr/share/doc/packages/hf,
the main text is this.


3) PTT Wiring :
   ============

As in evey ham radio digimode program, you have to connect computer and
transceiver with 1 npn transistor und 2 resistors:

                              _____
                      /------|_10k_|---O  
 RTS pin             /                PTT pin 
 Computer           /                 Transceiver
       _____    B |/      C
 O----|_10k_|-----|\
                  |  \_|  E
                      \
            npn        \
            Transistor  |
                        |
                        |
               gnd    =====
                                               

Naturally there are many more sophisticated possibilities 
with opto couplers and relais...


4) Problems with start of hfkernel :
   =================================
    
Sometimes the program 'hangs' while starting, stop it with 
<strg> c or with hfkernel -k and try again.

Failure of start script hf, or do you want to start manually:
Try to start first hfkernel as root from console:
# hfkernel
and if successfull
(corrout	...	intermediate   ...)
add the other necessary options one by one.
Most important is the socket to hfterm (-c /var/run/hfapp)
and the serial line.
Then start as normal user under X:
# hfterm -k /var/run/hfapp 

Most common error messages:

Error containing 'MMAP': (MOST COMMON ERROR !)
Soundcard and / or driver does not support 'mmap()'
(means DMA, program handles sound card buffer directly).
Try first the experimental option -n (no mmap)
or try option -h (force halfduplex).
More help: See 7).
The best way is: Set the options in /etc/hf.conf with your favourite editor 
and then start again
# /usr/bin/hf

Error containing 'open':
Is the sound driver loaded? (see 7).

Error containing 'ioctl: TIOCMBI[CS]:'
Is the serial line correct in /etc/hf.conf ? 

Error containing 'permission denied':
Error containing 'pthread_create':
If starting hfkernel as non-root:
Is the suid bit set?
# ls -l /usr/bin/hfkernel 
should result something like:
-rwsr-xr-x   1 root     root       372757 Sep 13 10:34 /usr/bin/hfkernel
The 's' is the suid bit, gives normal user super-user-rights when
executing it.
If not there, set the Suid Bit by hand:
# chmod 4755 /usr/bin/hfkernel .
There are 2 other alternative workarounds for the non-root users, 
see 6) Root Rights.

Depending on some combinations of soundcard / sounddrivers,   the
hfkernel can hang in an endless loop with  "... fragments passed since
last wakeup" or  "OSS driver lost interrupt".  Sorry, there were some
crashes ! In Version 0.6 I built in a brake to prevent this, so hope it
helps.  More about soundcard / sound driver problems: see 7).

In half-duplex mode, if the numbers
corrout ... intermediate ...
always big, (about 8000),  maybe the option
'-R' (disables RDTSC instruction, processor dependant, often with
non-intel architectures) has to be added.

If pactor/amtor ARQ works not or bad later, (especially if hfkernel only runs 
with the option -h, or -still worse - -R,) you have calibrate, see 8).  

Btw, try to get hfkernel running with the script hf, which includes 
/etc/hf.conf. All additional options can be put in /etc/hf.conf into the line 
"otheroptions", separated by spaces, e.g. "-n -i ".

Here is the complete list of hfkernel's options:
  -2: standby: disable monitoring of 200baud signals
  -3: standby: disable monitoring of 300baud signals
  -a: audio device path (default: /dev/dsp)
  -c: path of the communication socket (default: hfapp)
  -f: standby: disable frequency estimation
  -h: force half duplex mode
  -i: invert PTT (default: PTT = positive signal)
  -k: stop hfkernel (e.g. for use by scripts)
  -l: logging (default: off)
  -M: mixer device path (default: none)
  -m: CPU clock in MHz (exact at the kHz level)
  -n: no mmap() (which some cards/drivers can't) (experimental!)
  -p: path of the serial port to output PTT (default: none)
  -R: disable the use of the rdtsc instruction (Intel systems only)
  -r: access permissions of the communication socket (default: 0777 = rwxrwxrwx)
  -s: soundcard sampling rate correction
  -t: gettimeofday correction factor

   
5) hfterm and the digimodes:
   =========================

Most of the program is very intuitive. Help with F1. The state menu,
mode menu, and the buttons will explain  themselves. If you don't like
mice, like me, the shortcuts will make you happy (see menues): modes by
F..., text macros by <Shft>F... 

There are at all 3 levels of debugging output:
1. the xterm running in the background: hfkernel's low level
2. the monitor window: the protocol level
3. the most useful for the user: the Status Display Window on the 
   bottom of the screen tells you what the program is just doing. 

If you want to know how the digimodes sound:
Examples in www.wunclub.com/sounds !

Hfterm thinks. (autorx / autotx feature). If you write into the tx
window, it switches to the fitting or last tx mode. After some seconds
it switches to rx, exept you had chosen a tx mode explicitely, e.g. RTTY
TX.  This autorx/tx is useful for the beacon and mailbox functions, but
can be disturbed by too few waiting and too much clicking. Especially
MT63 has long delays between rx and tx and back because of its data
interleave! Be patient! But I like it, it makes me relax. If the program
hangs, first wait, then maybe you have to restart (hf, not the computer.
We do not have anything to do with Bill.)

 "SPECTRUM DISPLAY"
If you are a new user, get into feeling with the program by looking at
the spectrum. If no signal appears, look if the tx and the soundcard 
(line out) are cabled correctly, and if the signal is adjusted well by a
potentiometer in the line or by a mixer like aumix (for OSS) or
alsamixer (for ALSA).
The spectrum has a blue and a red line for mark and space, and in the
bottom there is a green line, the squelch.
You can change mark and space frequency with the mouse: 
If one of the given shifts is chosen,  the left button will select the
mark (and space will be mark plus shift), or the middle button
will select the middle frequency. If the "other" shift is chosen, you
just have to click 2 times and select first mark, then space, and the
shift will be calculated. The values are stored.
The right button selects the squelch, it will not be stored, the start 
value will always be 10 %.

 "RTTY"
For founding first friendship with the program try to receive and later
transmit RTTY. Usual values for shift: 170 Hz and for baudrate: 45. 
Tune on your tx e.g. in the 20m band and look for the typical  beautiful
machine-sound of rtty. Open the frequency spectrum display through the
menu 'utilities' or - quickest - by keying F2 or 'Ctrl  F' (keep Ctrl
pressed and hack on F). In my experience it is quickest not to change
mark and space in the program, but turn the tx until the double peak of
an rtty QSO  come under the blue and red lines in the spectrum. By 'Ctrl
F' again leave the spectrum. You will see the Receive window decoding
text. If someone is calling CQ - if you have an amateur radio license -
why not answer? Maybe you have to adjust the output level of your
soundcard,  do not tx with too much power (20 W is enough!), listen
before tx... like in all the digimodes. If you hold on transmitting,
there comes a "diddle" with "LTRS...".

 "PACTOR - AMTOR - GTOR"
Pactor is a science...  If you do not know what Pactor is, read 
'pactor.txt' in /usr/share/hf (sorry, German only.) First train to
receive. It is more difficult than RTTY. In the spectrum Pactor looks
almost like RTTY, but the sound is  hacked in typical 1.25 sec rhythm,
(Amtor: 0,45 sec), sometimes you can hear the short control signals  of
the other station in between.  Even if you adjust frequency exactly,
sometimes you will not decode anything  because people are using Pactor
level II and maybe III with a PTC.  Sometimes you fish a callsign out of
the fog, which is sent in level I. 
Call CQ by FEC. Be patient! If you have a partner answering by FEC, try
to call him by ARQ. Call a mailbox by ARQ.  After transmitting, do not
go to standby directly, the  program might hang. Instead, do QRT ('Alt
Q'), wait a little (let the program finish its QRT routine!), then it
will  go to standby by itself.

 "MT63"
Very much thanks to Pawel Jalocha, SP9VRC, who made this great new mode, 
which is said to be very robust, especially at bad conditions better than
Pactor 3, and might be precious for future "emcomm" systems.
(Emergency Communication, see the linlink mailing list at www.wetnet.net, 
there is an interesting discussion about this topic at the moment.)

Because of the many interesting experimental possibilities  I inserted
Pawel`s complete MT63 package with the console programs  mt63tx, mt63rx
und mt63trx, the tools morsecod, peakrms, addnoise  and the calibration
program ratecal1 into the hf package. Thanks! 

I insert here a part of Pawel's documentation on MT63.
(The complete text is mt63.txt in the doc dir.)

This is the release 0.5 of the MT63 modem for LINUX.
Date: 08-JUL-2004, Author: Pawel Jalocha, SP9VRC, Pawel.Jalocha@cern.ch
Important notes:
1. To take full advantage of the MT63 modem, the sampling rate of your
   sound card needs to be either calibrated or dead precise on 8000.0 Hz.
   For now, the MT63 receiver doesn't tell you the rate mismatch, I may provide
   such facility in the next software versions.
2. When connecting the audio to your PC/laptop be carefull not to damage
   the computer audio inputs. I suggest you first connect the grounds
   of the radio and the computer and only then connect or disconnect
   the audio cables.

The MT63 modem is intended for amateur radio as a conversation (RTTY
like) mode where one station transmits and one or more other stations
can listen. In short, the modem transmits 64 tones in its 1 kHz
bandwidth: the audio range for the tones is 500-1500 Hz. The
differential bipolar phase modulation is used to encode 10 bits of
information per second on each tone. The user data in the form of 7-bit
ASCII characters is encoded as a set of 64-point Walsh functions. The
bits are interleaved over 32 symbols (3.2 seconds) to provide
resistance against both pulse and frequency selective noise or fading.
The character rate equals to the symbols rate thus the modem can
transmit 10 7-bit characters per second. This modem can as well run in
two other modes obtained by simple time scaling, the possible modes are
summarized here:

Bandwidth  Audio range  Symb. rate  Char. rate  Interleave/char
  500 Hz   500-1000 Hz     5 baud     5 char/s   6.4 or 12.8 sec
 1000 Hz   500-1500 Hz    10 baud    10 char/s   3.2 or  6.4 sec
 2000 Hz   500-2500 Hz	  20 baud    20 char/s   1.6 or  3.2 sec

For each mode the interleave factor can be doubled thus each character
becomes spread over twice as long period of time. The first experiments
with this mode were done on the EVM56K DSP evaluation board from
Motorola and the package was named MT63ASC.ZIP. This LINUX
implementation is written to be compatible with that package. The MT63
modem is made for single side band operation. The audio generated by the
modem (sound card output) is applied to the SSB modulator. On the
receiver side, the output of the SSB demodulator is put into the sound
card input. The envelope of the MT63 signal is not constant as in other
multi-tone systems - it is rather noise-like. One must be carefull not
to overdrive the transmitter. The receiver of the MT63 is self-tuning
and self-synchronizing thus the radio operator is only required to tune
into the signal with +/- 100 Hz accuracy for the basic 1000 Hz mode. The
modem will tell the actuall frequency offset after it is synchronized.
The operator should not try to correct this offset unless he is able to
tune  his radio receiver very slowly, because the MT63 as a low rate 
phase modulated system does not like sudden frequency changes. Pawel,
SP9VRC. (Thanks Pawel!)

I made in hf the "flushing data interleaver" at the end of each transmit
cycle still longer than Pawel in his original program,  to prevent my
squelch function from cutting away the last data at receive.

 "PARAMETERS"
By 'Ctrl P' you will open a notebook where you can enter parameters like
mark, shift, baudrate, txdelay etc. for the different modes. When you 
finish the program, the parameters will be  stored in ~/hf/hfterm.rc
(digital, please do not edit!).

 "BRAG" 
also called 'Personal Data': By 'Ctrl B' you can open an entry field
where you can enter your call and personal 'brag' e.g. power, rig, 
antenna, locator, qth, name, mail address. This data will be used by the 
text macro processing functions to create texts while transmitting. When
you finish the program, the brag will be stored in  ~/hf/hfterm.brag, so
the everlasting fame of your amateur radio station is granted on your
hard disk. 

 "FIXTEXTS"
If you like text macros - edit your Fixtexts (up to 12) by 'Ctrl T'.
There are macros allowed for your call, name, QTH,  locator, rig, pwr,
ant, mail/internet address, timestamp and all the dates  of your QSO
partner, and: [B] for beacon - e.g. CQ. . You can not keep this all in
mind?  No matter, the window on the bottom reminds you.
If you want to transmit one of the macros, click on one of the 12
buttons, or 'Ctrl F<1..12>', lean back and drink coffee.  The fixtexts
are stored in ~/hf/fix.<nr>, they can be edited there. By renaming or
inserting you can import here any text, maximum 1024 bytes.  You can
also choose a file to transmit from file menu.

 "LOGBOOK"
We even have 2 logbooks. One for big screens, always open. Write your
entry, and click 'save' in the log menu to save, or 'clear' to clear. 
The other for my little old screen (a bigger one does not fit in my
shack).  Shrink the big log frame by 'Ctrl Q'.  By 'Ctrl N' you can open
the little log frame if you need it and you will still see the status
display on the bottom. 'Ctrl L' shows the log list. 'Ctrl A' archivates
the log file and starts a new one.  With 'Ctrl O' you can search old
entries to  edit or clear them. The log will be stored in portions of 50
entries, you will be able to  change this number from next version on.
The old log files are numbered and can be opened with every text editor.
The ascii format is adapted to cabrillo standard.

 "RX-TEXT"
If you receive something, it will be stored, the old stored rx files
will be renamed, and kept, up to 5 of them, in ~/hf. In the file menu,
there is also an option to choose a file for storing.


6 ) Superuser Rights :
====================== 

The problem is, hfkernel needs superuser rights because of its real-time
high-priority modus. But you should run programs if possible only as a
normal user.  To let non-root-users start hfkernel together with hfterm
easily, at installation hfkernel gets superuser rights by the suid bit: 
chmod u+s /usr/bin/hfkernel 
Then normal users can execute the program with superuser rights.
If this is o.k. and running, you do not have to read on this chapter.

If not, in  /etc/hf.conf 2 other ways are prepared and described. You
need 1 of the (2 or more?) programs that make normal users able to run
programs with root rights: 'su1' or 'sudo'.

su1: (see 'man su1')
Install it, open /etc/su1.priv whith an editor 
(of course, as root) and change like below:
	# Define some privileged users
	define	GODS	root <your_user_name>
and
	# These never require a password since they are reasonably safe
	ask	never
	allow   GODS    prefix  /usr/bin/hfkernel
Now, hf (started by you as normal user) calls hfkernel (with superuser rights).
An example file for /etc/su1.priv is 
/usr/share/hf/hfsu1.priv.txt
  
sudo: (see man sudo)
First '/etc/sudoers' has to be edited as root.
Insert the line:
<your_user_name> ALL=(ALL) NOPASSWD: /usr/bin/hfkernel
Now, hf (started by you as normal user) calls hfkernel (with superuser rights).

Be careful: All of theese workarounds can be security holes!


7) Problems with sound cards / - drivers :
   =======================================

Soundcards:
===========

hf does not run with every soundcard and / or driver.  The reason is not
bad code, but good code: Because of the time-critical ARQ protocols it
is necessyryx to work in real-time mode. For this, the author Tom Sailer
choose the best way and wrote the program with the "mmap()" system call.
This is also called "DMA" (direct memory access) and means mirroring the
soundcard's ring buffer directly into the working memory of the computer.
And not every soundcard can do this, and not every driver supports this.

I am experimenting with an alternative code with the OSS API using
normal read()'s and write()'s instead of mmap(). For this the option
"-n" (no mmap) is prepared.

The soundcard must be supported by the OSS driver. It must support 16bit
sampling in the endianness of the CPU, 8kHz sampling rate, memory
mapping of the DMA buffers and triggering. It must be able to work in
"mono", which some new cards cannot do any more! An improvement is planned.
planned.A full duplex soundcard is preferred.

On half duplex soundcards, the soundcard must be switched if the
protocol changes from receive to transmit and vice versa. This lasts
quite long; anywhere in the region of 5 to 35 ms. The program measures
an average at startup. It tries to hide this latency under the PTT keyup
delay (TXDelay), so set the txdelay to a larger value! And hope that the
propagation time to your peers plus their txdelay is also longer.

The audio levels and parameters may be set with the usual mixer tools.
Builtin AGC should be disabled.

Problems (crashes & hangs, as described in 4) ) have been reported with:
ESS 1968: "chipbug", does not work with ALSA and not with OSS drivers.
VIA82C686: works not with OSS driver. But works with ALSA. 
Intel 101k: many xruns ("OSS driver lost interrupt"), bad decoding.

Drivers :
=========

hfkernel has been written for the (old) OSS drivers, but it often runs
(often better) with the new ALSA drivers, as they have an OSS emulation
which is improved steadily. Depending on the card, hfkernel runs with
none, one or both of these classes of drivers.

OSS:  

In old Linux distributios there was a 'free' packet (paid with the 
distribution), e.g. in my old SuSE 6.3 in /tmp/opso, it works good with
hf, I am still using it. As far as I know, OSS is still being
maintained. The new OSS drivers are not free any more
(www.opensound.com).

In old and also in new (up to kernel 2.4...) linux versions there are 
kernel modules included, similiar or same to the old OSS drivers. 
(Unlike the ALSA modules, they do not begin with 'snd-'). They can be
found in /lib/modules/<kernelversion>/misc. (Source in
/usr/src/linux/<kernelversion>/drivers/sound). You can install them with
modprobe (after unloading ALSA by rcalsasound stop). See man modprobe,
modprobe -p <module> (for parameter info),  modprobe -d <module> shows
which other depended modules are loaded before. You will get help for 
ISA-PNP cards by pnpdump and isapnp, (see manpages) and for PCI cards by
lspci, and you find docs in  /usr/src/linux/Documentation/sound for your
sound card.

ALSA:

At this time the new ALSA (Advanced Linux Sound Architecture)
is being developed quickly and will be used by many linux systems.
It will be standard part of the new kernels from 2.6 on.
Latest version of ALSA and lots of good documentation at:
http://www.alsa-project.org
http:///www.alsa-opensource.org
Mailing lists can be subscribed, almost 1000 mails a month:
https://lists.sourceforge.net/lists/listinfo/alsa-devel
https://lists.sourceforge.net/lists/listinfo/alsa-user

The ALSA system has 2 ways of OSS emulation: 

The default one (which is in the kernel modules)  is activated when a
programm like hfkernel approaches the first (2nd ...) sound card as
/dev/dsp0 (1 ...), as all OSS programs do.

The other, alternative one is an additional library, which will linked
to the program by the script  /usr/bin/aoss (contains: env
LD_PRELOAD=/usr/lib/libaoss.so $*) e.g.: aoss <program>. This is
prepared in /etc/hf.conf.In order to use this lib, you have to prepare a
config file /etc/asoundrc or ~/asoundrc containing (if you have one
sound card)

	pcm.dsp0 {
		type hw
		card 0
	}
In newer ALSA versions this is done already.

Both ways of ALSA OSS emulation are still in development, and because
hfkernel's special desires (mmap and realtime mode) it sometimes does
not run with ALSA, depending on the sound card! The guys are working
very hard on improvements, so please try to get  the latest version of
ALSA and be patient with your patient  because your patient is a patient
patient.

EXTENDED PROBLEM SOLVING STRATEGY :
==================================

If hf does not run, I suggest the following:

for (;;) {
	Backup your data.
	Try first with the soundcard and driver you have. Then:
	Try option -n (no mmap) or -h (halfduplex).
	Try your ALSA in native mode.
	Try ALSA with aoss prefix (as is prepared in /etc/hf.conf).
	Try latest ALSA version, its becoming better.
	Try latest ALSA with aoss prefix.
	Try commercial OSS, if you have (maybe free in an old distribution).
	Try OSS kernel modules which are in your distribution.
	Try another soundcard.
	if (nervous system == corrupted) {
	break;
	}
}
if (you can program in c) {
	Improve l1/user/oss.c: oss_nommap_fdx_driver() and 
	oss_nommap_hdx_driver(), there I try simple read()'s und write()'s 
	with the OSS API. (I, Gnther, am working at this, hfkernel can 
	do bad-quality pactor ARQ with option -n  already! 
	Please contact me via mailinglist!)
	Or: re-write ist into ALSA API, which has more possibilities for 
	mmap-like operations. (They told me, it is more difficult than
	OSS API) (If you do this, use newest ALSA lib, the API changed a bit 
	at Version 1.0)
}
else {
	try the many good psk31, MT63, MFSK programs 
	that run well and make fun,
	try hf 1 year later...
}

If hf runs, fix it with glue and never touch your system again. Never
work more than 24 hours an evening. Look if your family is still there.
And do not forget: Do it only with fun. If depressive, mail me. I am a
psychotherapist. Real. It made me depressive sometimes, the whole thing, 
I healed myself by playing piano and doing other things, and by
celebrating computer-free days.


9) Real Time Problems and Calibration :
   ====================================
   
HF protocols are usually synchronous. They require an exact clock source
to remain bit synchronous even during longer disruption of the
propagation. SITOR (similiar AMTOR) for example specifies that the
reference clock must be no more than 20ppm off the ideal value. It's
difficult to find such an exact clock source, therefore all the options
chosen by the current implementation require manual tuning.

If the soundcard is full duplex capable, the reference clock is derived
from the sample clock. To correct for inaccurate sample rate information
given by the OSS driver, the -s option can be used. Your soundcard
should use real crystals instead of cheap ceramic resonators.
Also MT63 in hf uses the sample rate correction factor (-s).

If the soundcard is not full duplex capable, the above method cannot be
used. On Intel systems, the program tries the RDTSC (read time stamp
counters) to see if it is available and working (on Pentium class
computers and better, this should be the case). These counters increment
at the CPU clock rate, therefore the CPU clock frequency must be known
to the program, accurate to the kHz level (option -m). Don't be fooled
by marketing gags, eg. an AMD K5 PR133 runs at 100MHz. 
 
On non-Intel systems or if the RDTSC instruction is either unavailable
or not working, gettimeofday is used, in the hope that the tv_usec field
is accurate enough. Systematic frequency errors may be corrected by the
-t option.

RTTY, Pactor-FEC and Amtor-FEC might work without calibration, 
there have also been reports about successful Pactor- / Amtor-ARQ 
connects without calibration. It depends on the accuracy of
your card`s samplerate! 
If it is bad, or the connects should last longer, however, a calibration 
improves the performance of the program. 
Also for MT63 a calibration is necessary in many cases.

As described above, with options hfkernel can be given correction
factors for the soundcard's sampling rate (-s),
the cpu clock rate (-m) and the gettimeofday funktion (-t).
-s only works for full duplex mode (time derived from soundcard),
-m only works for half duplex mode with working RDTSC instruktion,
-t only works for half duplex mode without RDTSC instruktion,
which is worse, as the prozessor gets time via gettimeofday(), which is
not so exact because of changing system load.

In the hf package there are 3 Programs for calibration. Two of them,
dcf77rx and reffreq, even write their results to the config file
/etc/hf.conf by themselves. But please test it before use, and maybe you
have to correct something by hand. In the end of this chapter I will
describe a very primitive estimation I succeeded with, and a script
which guesses the sample rate for MT63. This helps for old hardware
where the cpu and / or memory might not be sufficient for the
calibration algorithms.

dcf77rx   
=======  
uses the timecode signal of the well-known German long wave reference
frequency transmitter DCF77 at 77,5 KHz near Frankfurt, that has a power
of 25 kW an can be received up to about 2000 km distance almost all over 
Europe. It gives, btw, the signal for the radio clocks. (See also
dcf77.txt, sorry only German.) If you never heard this signal, just try
the demo program dcf77gen !

The signal must be converted to a frequency of 1000 Hz and put to the 
Mic- or Line in - input of the soundcard. Tune your receiver to 78.5 kHz
LSB (oder 76.5 kHz USB). I had no LF receiver, so i got inspired by
ideas from the internet and home-brewed a little converter that makes 
4077,5 KHz from the 77,5 KHz, which I can receive with my rig.  I needed
only a 4MHz crystal from the box, 5 npn transistors and some more parts
which need no exact values. To receive an old ferrite from a radio or a
wire is o.k. The circuit is in /usr/share/lfconv.jpg. 

Start the dcf77rx program (preferably as root). In SSB or cw mode you
have to tune exactly on 1000 Hz, until you see the second ticks being
recognized as regular as possible. The option -v 2 or (after some
training) the rotating rod will help. 

After 1-2 minutes (under error free conditions), the program should have
aquired the DCF77 time. From then on wait about 15 minutes so you can check 
the results. You can write them down for sure. But they will be written 
automagically to /etc/hf.conf. 

But please see there if the correction values seem 
credible and constant, and edit them if necessary!
The last value of each kind hf.conf will be read by the 
start script hf and used as options for hfkernel.

dcf77rx also has an option to set the system time (see man dcf77rx).

dcf77rx and also dcf77gen already have the -n (no  mmap) option. It gave
acceptable values for the most important correction  parameter -s for
me. The others vary. 

The swiss timecode transmitter HBG at 75 kHz might probably be used,
but we do not know its exact transmission format (it seems to be
very similar to DCF77).

If you want to tune 2 sound cards together, also over radio, 
one partner could run dcf77gen and the other dcf77rx.


reffreq
=======
If you cannot receive  DCF77 or HBG, you can use the reffreq
binary and a known exact clock source in the 20Hz-20kHz region. 

If dcf77rx works bad, but if you can receive the DCF77 signal in AM
(2500 Hz), run reffreq -f 2500, it pulsates, but it works!

If you are looking for a reference frequency on the radio,
hfterm's spectrum can help you in raw measuring the frequency.
Guess it is a straight value, you can then use reffreq with it.

Thanks to Dave <dalechid@cox.net> who gave the good info on the US time
standard station WWV, Boulder, Colorado,  which transmits 400 and 600 Hz
in an alternating way at  2.5, 5 and 10 MHz AM! So run reffreq -f 400 or
reffreq -f 600. Who knows more about JJY in Japan?

A readily available in most households and usually very accurate source
is the line sync of an ordinary TV receiver. The line sync of the second
public german TV channel (ZDF) is used as a reference frequency even by
official bodies. 

Tune your TV equipment (with baseband video output) to a TV channel and
feed the video output to the soundcard. Run  "reffreq -f 15625" as root.
After a few seconds, the program should have calculated the correction
parameters. (The command line above implies PAL format with its 15625 Hz
line sync frequency. For other video formats, use the appropriate
frequency.)

In CQDL 3/2003, page 168, there is an interesting article by Stefan 
Steger (sorry, German only), describing how to get the line frequency
signal from a TV, with a circuit that derives reference frequencies.
(http://home.t-online.de/home/stefan.steger/homepage.html)

With a scope I could get the 15625 Hz from the cable that connects a
satellite receiver with the TV.

Reffreq could serve for calibration over the air:  One sets in hfterm
mark and space to 1000 und sends RTTY (RTTY TX im Mode menu), Funkgert
auf AM.  with the transceiver at AM. A sinus tone of 1000 Hz will be
transmitted. The other hears the tone with AM und calibrates with
reffreq -f 1000.


ratecal1:
=========
This tool is again by Pawel and part of his MT63, thanks!
I insert here Pawel's instructions:

The MT63 is a synchronous system and it relies on the sampling rate
to be the same at the receiver and the transmitter. At least the sampling
rates should not be different by more that 10^-4. MT63 samples at 8000 Hz
thus if your card runs at 8000.5 it's probably OK but if it runs at
8005 Hz it's not good ! 

An extreme example can be my Soundman-16
(PAS-16 clone) which when asked to run 8000 Hz tell me, that it can only
do 8008 Hz and it reality it runs at 7910.3 Hz which makes an error
of more than 1% - far too much for the MT63 even at infinitely good S/N.
My other two cards (DSP-16 and Ensoniq 1371) are more reasonable: they
show an error of 0.3 to 0.5 Hz at 8000 Hz sampling.

To measure the sampling rate I have prepared "ratecal1" utility.
The program has been tuned to work with the HF timing standards
which transmit short pulses each second. I used the signals on
4996.0, 9996.0 and 14996.0 kHz. I set the HF receiver in the USB mode
and 1.0 kHz below the frequency to get pulses at 1000 Hz audio.
In principle any signal having a periodic envelope
with a known period can be used, so an AMTOR/SITOR station is OK,
if you change the reference period to 0.450 sec with -T0.450 option.
What counts is the envelope of the signal around the frequency
given by -F and within the bandwidth given by -B.

To measure the sampling rate I tune my HF receiver to say 4995.0 USB
so the pulses come up as a 1 kHz beeps. I connect the audio to the given
soundcard (say /dev/dsp) and I type:
ratecal1 -d
After 10-20 seconds I can see the measured sampling rate of my card. 
The calibrator will try to match the delay which gives the smallest
differential error. This procedure ussually gives you the rate
within 0.5 Hz (at 8000 Hz sampling).

For more accuracy multiply the repetition time: for example
give -T4.0 thus 4 second reference period; a signal with period
of T is as well periodic over time = N*T where N is an integer.

If your sampling rate error is large (like with my card)
tell the program the (about) true rate with the -R option.
for example for my card I run the more precise test:
ratecal1 -r8000 -R7910 -T4.0 -I40 -B200
The most precise test I have ever done was by typing:
ratecal1 -r8000 -R7910.3 -T300 -I60.0 -B500 -D4
and leaving it for 10 minutes on the 14995.0 USB.
After the calibration is done put down the two values:
the rate you request (-r) and the true rate you get.
You may try different requested rates (-r) so you get
the real rate close to what you like.
For example with my card, I need to request 8100 Hz to get
real rate of 8018.2 Hz which is closest to the 8000 Hz
which I want for my applications.

Timing signals on 4996, 9996 and 14996 kHz are not ideally periodic
because some pulses are longer, some are doubled, etc.
This has certainly negative effect on the calibration result.
The pulses there are not transmitted all the time but according
to certain schedule thus we can not apply too long integration times too.
Still rate measurement with an error about 10^-5 is possible.

When you know the true sampling rate you can correct for this
when working with the MT63 mode with the -R option. Say, you measured
that you card at /dev/dsp2 samples at 8010.5 Hz, so you type:
mt63trx -d2 -R8010.5
I do so with my PAS-16 clone and it works just fine for the MT63
despite the 1% error.
You may happen to have a card which refuses to run at 8000 Hz - but you
are not all lost. Say the card at /dev/dsp1 only likes the 9600 Hz
sampling and when asked for 9600, it samples at 9605.4 Hz (you measure
this with the ratecal1). To run the MT63 receiver you type:
mt63rx -d2 -r9600 -R9605.4
the mt63rx will ask the sound driver to sample at 9600 and knowing that it
really means 9605.4 it will apply the proper rate converter.
Ideally every operator should measure the sampling rate of his soundcard
and then correct for this when transmitting and receiving. This way every
signal on the air has same absolute timing and thus can be read by anyone.
(Thanks, Pawel!)

I (Gnther) compared ratecal1 with the 2 other tools and found the
same good results.
Pawel told me that the mentioned pulsed signals at on 4996.0, 9996.0
and 14996.0 kHz come from Russia. I could not receive them at time of my test,
but used the signal of the dcf77 beacon, which is also pulsed.
If I divide the `official` sample rate of my soundcard by
the real sample rate which is obtained by ratecal1, I get the same 
"soundcorr" as by the 2 other tools, exact to a degree of 10^-4 which is o.k.
Also ratecal1 could serve for an on-the-air-calibration: 
One sends Amtor or Pactor ARQ 
or even dcf77gen (but: this still can not take a correction as option!),
the other runs 
ratecal -d -T0.45 (for Amtor) or -d -T1.25 (for Pactor)
or -d for dcf77gen.

I had an adventure with my laptop (150 MHz Pentium 1, 14 MB RAM only!),
I must tell you. It may help you if you also have old hardware...
Hf and also MT63 works with the laptop, but the calibration tools work bad, 
they need much men and cpu! It had worked in my tests with 2 cabled 
computers with Pactor and Amtor, but no single sign decoded with MT63.
I had no idea about its real sound rate!
I wrote the script ratetry (look at it's text, you may modify it to
your needs, it just tries transmitting with rates from ...to..., 
given by option.) I first checked it with around 8000, no success.
In my despair I found another, very primitive first-look test,
working with my ears only: I set both computers to
mt63tx -d -C_____T____
which means, send MT63 with a single dah (T) within 9 spaces as cw id.
Now I heard the difference: The laptop was much quicker, it overtook
the other computer. It really sounded like the little fox jumping 
over the lazy dog, I could hear it!
Now I guessed the real rate of the laptop to be 8100, 8200, 8300...
and ran MT63 on it with 
mt63tx -d -C_____T____ -R8100 and so on.
With R8300 the two 'dahs' sounded in about equal rythm.
So I could use ratetry 8200 8350 and could receive the data with
the other computer between laptop's "real rate " 8241 and 8335!
So, now lab working, developement can go on.


9) Mailbox :
   =========

hfterm can be linked by TCP/IP (a telnet-Port)  with F6FBB, which is
well-known and proven as Packet Radio BBS. It can be installed on the
same or an other computer linked by ethernet or PLIP. If your computer
ip adress and Port for F6FBB differ from the defaults (Localhost =
127.0.0.1 = the same computer, Port 6300 = default for F6FBB) change
the corresponding entries in /etc/hf.conf.

At first there is something to change in only 5 configuration files and
one binary. Don't fear, I help you. Log in at a console as root.  If
xfbbd is running already, (test with ps -ax), quit it with killall 
xfbbd. For test after the changes start it again by xfbbd.

The start is quite easy. 
/etc/ax25/fbb/passwd.sys:
Replace the password after the line 
#PASSWORD OF ALL NON DEFINED CALLSIGNS.
by an empty line!

/etc/ax25/fbb/port.sys:
Changes marked by arrows.
# FBB7.00
#
#Ports TNCs
 1     1				<-----	
#
#Com Interface Adress (Hex) Baud
 8   9         189C          0		<----- (189C hex. = 6300 dec.) 
#
#TNC NbCh Com MultCh Pacln Maxfr NbFwd MxBloc M/P-Fwd Mode  Freq
 0   0    0   0      0     0     0     0      00/01   ----  File-fwd.
# 1   8    9   ax0    250   3     1     10     30/60   XUWYL Linux
 2   8    8   0       250   2     1     10     0/60   TUR  # hf_telnet   <-----
#
# End of file.
#
(This port.sys is configured only for hf. If already other ports / TNC's
are there, for hf 1 Port / 1 TNC has to be added. The other marked lines
have to be inserted additionally for hf.)

/etc/ax25/fbb.conf:
(in older versions this was init.srv)
#
# FBB Set-up file
#
version = FBB7.01
callsign = DL4MGE.BAY.DEU.EU  			<-----   
# Qra Locator of BBS
locator = JN58NG				<-----
qral = JN58NG					<-----
# Qth of BBS
city = Baindlkirch				<-----
# First name of SYSOP
name = Guenther					<-----
# Callsign of SYSOP
sysop = DL4MGE					<-----
# Callsign (and route if needed) that will have copy of SYSOP messages
sysmail = DL4MGE				<-----
# Wait for informations (Name, HomeBBS, Qth, ZIP)
askinfo = NO 		(chose one of them)	<-----
#askinfo = OK		( "    "   "  "   )	<-----				
# First connection mask :
# 0  : Disable
# 1  : Excluded
# 2  : Local					
# 4  : Expert	
# 8  : Sysop
# 16 : BBS
# 32 : Pagination
# 64 : Guest
# 128: Modem					
# 256: See-all-messages
# 512: Unproto list asking is allowed
# 1024: Liste des messages nouveaux. List of new messages.
# original was: mask = 3616
# orig + 2 + 128 means: local and modem     	
mask = 3746					<-----
The most important is "mask" - add 130 to the existing "mask". This
means: 2 =  Local and 128 = Modem. So a new user can log in "local"
(this is also IP and telnet) and has access to the "Modem", which is 
also a TNC, and hfterm is seen as TNC by F6FBB. So a new remote
users can send mails from the first login on.
Btw, 3746 is  2 + 32 + 128 + 512 + 1024 + 2048.
It works also with mask = 130 which is 2 + 128!

/etc/ax25/fbb/lang:
Replace english.ent by an empty file.

/etc/ax25/fbb/lang/english.txt. 

You may optimize the messages for shortwave: shorten them! Now I found
out how to change this file without making F6FBB crazy: It is just
important that the number of lines remains 287! Lines beginning with a
comment sign (which is the '#' like in shell scripts) are not counted
for this. So you can just outcomment a line (don't delete it, this helps
you in  debugging later...), then copy it and edit it. The '$W' means
that F6FBB prints a newline.
Some lines (where F6FBB asks for name, home BBS, town etc.) depend on 
your "askinfo" in /etc/ax25/fbb.conf: newlines, if  "NO" (complete 
empty lines result in errors) or e.g. "Name ? ", if OK.
I will deliver my english.txt in /usr/share/<doc>/hf, just look
at it, edit it to your desires, copy it (as root) to it's place in
/etc/ax25/fbb/lang.

And now the most dangerous operation. You have to study medicine for 
this. Did you already hack a binary? It is simple! It is about F6FBB's
first login message, which is hard-coded, but does  not fit for our hf.
If a new user logs in at F6FBB, it says 
Login in read-only mode. 
You may leave a message to SYSOP.
But this is better, clearer:  
First login to register. 
You can send mail at next login!
How to do it? At first, in every case make a backup of /sbin/xfbbd. (The
binary can be destroyed by one sign too much!) Then open xfbbd  with an
editor, e.g. mcedit, best in overwrite mode. Look (F7 in mc) for
'SYSOP'. Overwrite the 2 greeting lines, not more. The first line has 24
chars, the second 33. For me it worked!

To configure F6FBB it sometimes kryptic, but it is a good and stable
program. I had to think and test much to get all this out. Thanks to the
friends from the F6FBB mailing list xfbb@f6fbb.org  and the good docs: 
http://www.f6fbb.org/fbbdoc/doc.htm  and, of course, for F6FBB itself to
its author Jean-Paul Robeller!

Now you can test F6FBB by `telnet localhost 6300`. It must be possible
to login with a call and register, logout, login again and to send and
read mails.

Now start of the hf-mailbox is simple: start als root xfbbd, then start hf
as normal user hf, then (if you want) activate the mailbox-beacon fixtext,
then activate the mailbox mit <Alt>M or with the State menu, and it runs!

The connection hfterm <--> F6FBB can now be tested by writing into the 
rx window (take Standby Pactor only mode, then you transceive in 
Pactor-FEC, or take MT63). So you can simulate the a remote user logging
in without having to cable 2 Computers or make a radio contact.

A Pactor connect to the call in menu Utilities/Parameters/Pactor/Mycall
or an amtor connect to the call in ../Amtor/Mycall opens the mailbox.
In FEC-modes the partner must call the call configured in 'Personal Edit` 
(brag). 

I tested the mailbox with 2 computers with cabled soundcards with Pactor
FEC, Pactor ARQ, Amtor FEC and Amtor ARQ with very good success.  With
RTTY and MT63, it runs now sometimes with a lot of good luck and 
patience, but it is still prone to errors, and F6FBB confuses.  (The
squelch function must have some delay, little for FSK and big for 
MNT63, so it lets through some rubbish at start and end of each transmit 
cycle.) So let's wait for the MT63 ARQ mode!

In use on the air as automatic mailbox station, concern the regulations
of your country! 

What about a ptc-free, microsoft-free, free-software based
mailbox at your Club station?


10) List of all programs in the package:
    ====================================

    All installed in /usr/bin:
   
    hfkernel,
    hfterm: 		the two main programs.
    hf: 			the start script for both.
    paccalc:		calculates pactor data throughput
    channel:		Short wave channel simulator.
    dcf77rx:		Calculates clock correction constants from DCF77 timecode,
			can also set system time, see man dcf77rx.
    dcf77gen:		generates DCF77 signal, see man dcf77gen.
    reffreq:		Calculates clock correction from reference frequency.
   			
			and: (new!) Pawel Jalocha's MT63 suite as 
			console programs, with many experimental features:
    mt63rx:		receives  MT63
    mt63tx:		transmits MT63
    mt63trx:		does both
    ratecal1:		calculates real sound rate from pulsed beacon
    addnoise:		adds noise to an audio file, for experiments
    morsecod:		converts digital to readable morse alfabets
    peakrms:		tools for examining mt63 audio files
    ratetry:		a script for testing mt63tx with different rates
			Help for all of them: Just call them without options
     

11) List of Help Texts :
    ====================

For now, there is help in English and German. Translations are very welcome!

The texts are in the doc directory of the program packet and will be
installed into /usr/share/hf. The most important texts can
be found in hfterm help menu (F1).

   
   ENGLISH:
   ========
   
   HF-HOWTO.txt		(this text, 
                	also with contents of man hfkernel and man hfterm)
   P-MB-list.txt	List of Pactor Mailboxes by Jost, ZS5S
   TODO			(the voices programmers hear in their heads...)
   english.txt		example for /etc/ax25/fbb/lang/english.txt.
   mt63.txt		Pawel Jalocha's original MT63 doc
   manpages:
   man hf, man hfkernel, man hfterm, man dcf77gen, man dcf77rx  	

   GERMAN:
   =======
   
   DE-HF-HOWTO.txt	(this HOWTO in German)
   dcf77.txt		(longwave time & frequency reference beacon)
   pactor.txt		(good explanation of pactor, H.-P. Helfert et al.)
   pactor.ps		(same in ps)
   manpages:
   man hfkernel-de, man hfterm-de   	

   PICTURES :
   ==========
   lfconv.jpg		circuit of DL4MGE's longwave 4 MHz converter
   

12) Conclusion :
================

There are still 'some' BUGS, also called FEATURES,  don't scream, better
wait, still better report, best help! (see TODO). I myself have been
inspired to learn c by this program, learned a lot (all I wrote above I
also did not know before!) and had (most of the time) much fun up to now!
  	
Ok, we hope Tom & you all will like ! So have a lot of fun ...

Gnther Montag DL4MGE

============================================================================

		
		AUTHORS

Thomas M. Sailer	HB9JNX/AE4WA	Mail: sailer@ife.ee.ethz.ch
			
Ralf-Axel Krause	DF3JRK		http://www.df3jrk.de.vu
					Mail: df3jrk@gmx.de	
					Packet: df3jrk@df0hmb.hh.dl.eu.ww

Gnther Montag		DL4MGE  	http://www.hfterm.de.vu
 					Mail: dl4mge@darc.de
 					Packet: dl4mge@db0zka.bay.deu.eu
						
Pawel Jalocha		SP9VRC		Pawel.Jalocha@cern.ch

Project homepage, latest news and download of latest version:
	https://sourceforge.net/projects/hfterm

PLEASE _DO _ subscribe the mailing list:
	https://lists.sourceforge.net/lists/listinfo/hfterm-hackers

please post all questions and bugs and tips to the mailing list:
	hfterm-hackers@lists.sourceforge.net
