Topics

VARA vs. Pactor-III Speed Comparison Plots

Tony
 

All:

I've spent several weeks evaluating the on-air performance of VARA and Pactor-III under various conditions. The variability of the HF channel can skew the results when comparing modems this way so the data was compiled from an average of several test runs before a comparison was made. The tests were conducted minutes apart in alternate succession with modem run times of 5 minutes or more.

Speed calculations were taken from Winlink sessions that include the total session speed and file transfer speed in BPM. Tests were conducted on the 40 and 80 meter bands where multi-path and noise tend to be at their worse. The plots below, which were compiled from more than 60 hours of on-air test data, reveal some interesting results.


The first plot shows that VARA can exceed Pactor-III speeds by a considerable margin under the best channel conditions. They also show that the performance gap falls off to the point where Pactor-III eventually becomes the faster mode as conditions deteriorate.   


The next plot shows the results for the 80 meter band. Once again, Vara shines under the best conditions while Pactor-III tends to do better with under worse conditions. As expected, the additional noise and multi-path result in slower overall speeds for both modems. Under the best conditions, speeds on 40 meters were about 1.8 times faster than those on 80 meters. 


The next image shows the 40 meter and 80 meter speed data on the same plot.  


The next plot reveals a more detailed look at the poor signal performance between the two modems. Pactor-III tends to be the faster mode when subjected to low signal-to-noise ratios which would suggests that the low-speed Pactor modes tend to be more robust. 


The next plot is more interesting from a practical point of view. It depicts the total Winlink session time i.e., the time it takes to complete a session from start-to-finish. Pactor-III took less time than Vara under the worse conditions while Vara completed sessions in less time than Pactor-III once conditions improved. 



It would be interesting to see how the on-air data compares to the more accurate data that's possible using the controlled environment of a path simulator. If anyone knows of such tests, please pass them my way.

Tony -K2MO

ALE
 


Hi Tony,

In the world of hardware HF Channel Simulator's, keep an eye out for a used Rockwell MDM-3001 MIL-STD modem, it has an excellent HFCHANSIM capability for your base band testing needs. Then too if you spot one cheap enough, the MDM-Q9604 modem has an ever better HFCHANSIM capability. I use both units here in my development and vakidation testing efforts

The MDM-3001 is about the size of your SCS P3 modem and is of a fan less design with a simple front panel LCD/button selection user interface. The MDM-Q9604 is a rack mount unit in part using an embedded Linux computer and hardware modem board and is fan cooled. It too has a simple front panel user interface but can also be controlled via keyboard and mouse with video monitor attached using a GUI interface.

Here is an old paper I wrote back when I first started working with the MDM-3001 describing its use at the time.

www.n2ckh.com/MARS_ALE_FORUM/Rockwell_MDM-3001_HFCHANSIM.pdf

/s/ Steve, N2CKH


At 05:15 PM 5/27/2018, you wrote:
Content-Type: text/html; charset=utf-8
Content-Length: 3448

All:

I've spent several weeks evaluating the on-air performance of VARA and Pactor-III under various conditions. The variability of the HF channel can skew the results when comparing modems this way so the data was compiled from an average of several test runs before a comparison was made. The tests were conducted minutes apart in alternate succession with modem run times of 5 minutes or more.

Speed calculations were taken from Winlink sessions that include the total session speed and file transfer speed in BPM. Tests were conducted on the 40 and 80 meter bands where multi-path and noise tend to be at their worse. The plots below, which were compiled from more than 60 hours of on-air test data, reveal some interesting results.


The first plot shows that VARA can exceed Pactor-III speeds by a considerable margin under the best channel conditions. They also show that the performance gap falls off to the point where Pactor-III eventually becomes the faster mode as conditions deteriorate.   

[]


The next plot shows the results for the 80 meter band. Once again, Vara shines under the best conditions while Pactor-III tends to do better with under worse conditions. As expected, the additional noise and multi-path result in slower overall speeds for both modems. Under the best conditions, speeds on 40 meters were about 1.8 times faster than those on 80 meters. 

[]


The next image shows the 40 meter and 80 meter speed data on the same plot.  

[]


The next plot reveals a more detailed look at the poor signal performance between the two modems. Pactor-III tends to be the faster mode when subjected to low signal-to-noise ratios which would suggests that the low-speed Pactor modes tend to be more robust.  []


The next plot is more interesting from a practical point of view. It depicts the total Winlink session time i.e., the time it takes to complete a session from start-to-finish. Pactor-III took less time than Vara under the worse conditions while Vara completed sessions in less time than Pactor-III once conditions improved. 


[]


It would be interesting to see how the on-air data compares to the more accurate data that's possible using the controlled environment of a path simulator. If anyone knows of such tests, please pass them my way.

Tony -K2MO

Tony
 


Hi Tony, In the world of hardware HF Channel Simulator's, keep an eye out for a used Rockwell MDM-3001 MIL-STD modem, it has an excellent HFCHANSIM capability for your base band testing needs.
/s/ Steve, N2CKH


Steve:

Thanks for the tip. It would be nice to find a hardware modem at a reasonable price. My guess is that it can accommodate soundcard modems as well as hardware modems. It might be tricky to get the audio levels right between the two, but I would assume the audio I/O can be adjusted.   

Hope you're doing well.

Tony -K2MO



All:

I've spent several weeks evaluating the on-air performance of VARA and Pactor-III under various conditions. The variability of the HF channel can skew the results when comparing modems this way so the data was compiled from an average of several test runs before a comparison was made. The tests were conducted minutes apart in alternate succession with modem run times of 5 minutes or more.

Speed calculations were taken from Winlink sessions that include the total session speed and file transfer speed in BPM. Tests were conducted on the 40 and 80 meter bands where multi-path and noise tend to be at their worse. The plots below, which were compiled from more than 60 hours of on-air test data, reveal some interesting results.


The first plot shows that VARA can exceed Pactor-III speeds by a considerable margin under the best channel conditions. They also show that the performance gap falls off to the point where Pactor-III eventually becomes the faster mode as conditions deteriorate.   

[]


The next plot shows the results for the 80 meter band. Once again, Vara shines under the best conditions while Pactor-III tends to do better with under worse conditions. As expected, the additional noise and multi-path result in slower overall speeds for both modems. Under the best conditions, speeds on 40 meters were about 1.8 times faster than those on 80 meters. 

[]


The next image shows the 40 meter and 80 meter speed data on the same plot.  

[]


The next plot reveals a more detailed look at the poor signal performance between the two modems. Pactor-III tends to be the faster mode when subjected to low signal-to-noise ratios which would suggests that the low-speed Pactor modes tend to be more robust.  []


The next plot is more interesting from a practical point of view. It depicts the total Winlink session time i.e., the time it takes to complete a session from start-to-finish. Pactor-III took less time than Vara under the worse conditions while Vara completed sessions in less time than Pactor-III once conditions improved. 


[]


It would be interesting to see how the on-air data compares to the more accurate data that's possible using the controlled environment of a path simulator. If anyone knows of such tests, please pass them my way.

Tony -K2MO
_._,_._,_

ALE
 


Hi Tony,

As far as the HFCHANSIM capability of the MDM-3001 or MDM-Q9604 is concerned, you can use it with any HF modem with accessible base band audio input/outputs. Which is to say that its complicated to say the least to make of it with a radio having an internal modem such as the TC4 installed inside my IC-F8101E or the internal USB codec in the same radio for software defined modem use.

The unit provides for front panel gain settings. You can configure in wiring for differential or single ended audio input/output operation while providing for your impedance matching.

For feeding multiple signals on the input to the HFCHANSIM as well as tapping the output to more than one place, I like the inexpensive passive and active mixer boards sold by Pricom.

http://www.pricom.com/Trains/PassiveMixer.shtml

The best results are always achieved when using a professional grade AC'97 sound device, where the least expensive COTS unit that I know of is the ASUS Essence II.

However I make use of a TI C5000 DSP developer board running my own firmware configured as a USB sound device. The same device being the target of my embedded modem development efforts. Many MARS members are using it for their PC Sound Device Modem with my MS-DMT software and firmware. It is only a $107USD item, which is really cheap as to pro-grade audio it provides. As a USB sound device its plug 'n play under Windows, Linux and MacOS as tested by myself aside from MacOS and my users on MacOS as well.

/s/ Steve


At 04:34 PM 6/6/2018, you wrote:
Content-Type: text/html; charset=windows-1252
Content-Length: 5484


Hi Tony, In the world of hardware HF Channel Simulator's, keep an eye out for a used Rockwell MDM-3001 MIL-STD modem, it has an excellent HFCHANSIM capability for your base band testing needs.
/s/ Steve, N2CKH

Steve:

Thanks for the tip. It would be nice to find a hardware modem at a reasonable price. My guess is that it can accommodate soundcard modems as well as hardware modems. It might be tricky to get the audio levels right between the two, but I would assume the audio I/O can be adjusted.   

Hope you're doing well.

Tony -K2MO



All:

I've spent several weeks evaluating the on-air performance of VARA and Pactor-III under various conditions. The variability of the HF channel can skew the results when comparing modems this way so the data was compiled from an average of several test runs before a comparison was made. The tests were conducted minutes apart in alternate succession with modem run times of 5 minutes or more.

Speed calculations were taken from Winlink sessions that include the total session speed and file transfer speed in BPM. Tests were conducted on the 40 and 80 meter bands where multi-path and noise tend to be at their worse. The plots below, which were compiled from more than 60 hours of on-air test data, reveal some interesting results.


The first plot shows that VARA can exceed Pactor-III speeds by a considerable margin under the best channel conditions. They also show that the performance gap falls off to the point where Pactor-III eventually becomes the faster mode as conditions deteriorate.   




The next plot shows the results for the 80 meter band. Once again, Vara shines under the best conditions while Pactor-III tends to do better with under worse conditions. As expected, the additional noise and multi-path result in slower overall speeds for both modems. Under the best conditions, speeds on 40 meters were about 1.8 times faster than those on 80 meters. 




The next image shows the 40 meter and 80 meter speed data on the same plot.  




The next plot reveals a more detailed look at the poor signal performance between the two modems. Pactor-III tends to be the faster mode when subjected to low signal-to-noise ratios which would suggests that the low-speed Pactor modes tend to be more robust. 


The next plot is more interesting from a practical point of view. It depicts the total Winlink session time i.e., the time it takes to complete a session from start-to-finish. Pactor-III took less time than Vara under the worse conditions while Vara completed sessions in less time than Pactor-III once conditions improved. 





It would be interesting to see how the on-air data compares to the more accurate data that's possible using the controlled environment of a path simulator. If anyone knows of such tests, please pass them my way.

Tony -K2MO

Graham
 

Hi Tony , More than 60 hour’s that was quite a feat !

 

There are some interesting points linked to the development of the  vara modem

 

The initial version, did deploy a change of protocol for the  lower speed  ranges , but,  with development , the  data  rates / s/n  of the  ofdm  modem ,  exceeded the  conventional  mfsk  rate/s/n.

 

Actually better  than 2fsk conventional rtty  , hence the  modem remains ‘single protocol’ ofdm  with  ‘low on air rate’ at all speeds  , with advantages of  noise rejection, selective fade , multiple path  over narrow b/w mfsk

 

Q:  When amtor  changes to  mfsk,  do you  note a higher level  of  power consumed ,ie , the supply current to the  Tx  increases , if so ,  what  change ?

 

Pactor  4

 

Q: Is it possible to run tests of the  vara  modem  against the  Pactor 4  modem, can anyone perform such a test ?

 

There  is a  interesting  ‘clash of  concept’ , with the  use of  high speed  serial  carriers , and the  low on air rate  multiple  carriers  of the  vara  ofdm  modem , to date , this  remains to be  explored …

 

73-Graham

G0NBD

Tony
 

Hello Graham:

It certainly was a time consuming process. It's interesting to note that short-term changes in propagation had less of an effect on modem speed than I expected. Speeds usually varied by 20 percent or less in the relativity short period of time it took to extract an average from multiple test runs.  

Jose did mention the changes he made to the protocol and I suspect it would have had a slight impact on the results. To your other question: no, I didn't measure variation in power consumption.

I'm not aware of any test data that includes Pactor-4 and VARA, but I suspect that would be something those in emergency communications would be interested in.   

Obviously, the proper way to compare HF modems would be under the controlled environment of an HF path simulator, but I think on-air testing can reveal certain throughput characteristics that may not show up on an artificial path. I think it all depends on how complex the real path is and what the path model limitations are.

Tony -K2MO 

     


 6/15/2018 10:37 AM, Graham wrote:

Hi Tony , More than 60 hour’s that was quite a feat !

 

There are some interesting points linked to the development of the  vara modem

 

The initial version, did deploy a change of protocol for the  lower speed  ranges , but,  with development , the  data  rates / s/n  of the  ofdm  modem ,  exceeded the  conventional  mfsk  rate/s/n.

 

Actually better  than 2fsk conventional rtty  , hence the  modem remains ‘single protocol’ ofdm  with  ‘low on air rate’ at all speeds  , with advantages of  noise rejection, selective fade , multiple path  over narrow b/w mfsk

 

Q:  When amtor  changes to  mfsk,  do you  note a higher level  of  power consumed ,ie , the supply current to the  Tx  increases , if so ,  what  change ?

 

Pactor  4

 

Q: Is it possible to run tests of the  vara  modem  against the  Pactor 4  modem, can anyone perform such a test ?

 

There  is a  interesting  ‘clash of  concept’ , with the  use of  high speed  serial  carriers , and the  low on air rate  multiple  carriers  of the  vara  ofdm  modem , to date , this  remains to be  explored …

 

73-Graham

G0NBD

Graham
 

Thank's for the  info Tony

Ah, clash of times,  the  new version has some  re-working  of the  lower speed's ,  a lot of  test data  has only been  made  available, since the  modem entered service , we could only  test over the  1000 mile  path  UK<>EA . The  latest HF version is  now frozen , good  reports of  maritime  mobile  EU<>VE  exchanges on 7 MHz , are posted in the  google 'vara'  group.

The power consumption   ofdm / mmfsk  is relevant, in the  sense , that  it affects the system  s/n moving  from  25  watts  average power ,  vara mode designed tx level , to  single carrier  , 100%  duty ,  raises the  level by  6 dB , but at  a disproportionately lower  data rate .

Work and testing continues for the  high speed  'FM'  variants ,  both  standard  2 meter FM  and  9600 baud replacement  modem  , topping out at  15K bps 

Take up on these versions  has not been  too  high ,  possibly as there point to point only , suited for trunking links , where as the  existing systems ,  1200/9600 baud are capable of  supporting  packet ax25l as well, due to frequency tolerance ,  there  only  suited to FM  transmission, where  the  tone  frequencies  are  protected  by the  modulation type. 

Pactor 4 , yes , that remains to  be  tested , if any one  can undertake  such a  test ,  the  results would be  quite interesting ..

73-Graham  

James_Clark@...
 

Tony: thank you *very* much for all the effort you put into the testing, it is exactly the information I have been looking for.

I spend a fair bit of time deep in the Australian bush and use Winlink/Pactor 3 frequently and based upon your tests I'll buy a copy of VARA to add to my armoury. Currently we only have one VARA base station in Oz (in Sydney on 40m) but I'm sure others will pop up in due course.

One spec. I have not seen is that for the minimum Windows machine on which to run VARA. As electrical power is often an issue for me I like to use my XP Intel Atom netbook when possible but I suspect it may struggle with VARA?

Tony
 

On 8/24/2018 10:42 PM, James_Clark@... wrote:

Tony: thank you *very* much for all the effort you put into the testing, it is exactly the information I have been looking for.

My pleasure James. It would be interesting to see how the two modems compare during your field tests. We have a large number of RMS stations here in the states so it was easy to test both VARA and Pactor via the same station.

I'd give VARA a try and see how things go with your notebook -- maybe someone on this reflector can verify whether or not VARA will run on an XP machine.

Look forward to hearing from you James.

Tony -K2MO

 






Tony: thank you *very* much for all the effort you put into the testing, it is exactly the information I have been looking for.

I spend a fair bit of time deep in the Australian bush and use Winlink/Pactor 3 frequently and based upon your tests I'll buy a copy of VARA to add to my armoury. Currently we only have one VARA base station in Oz (in Sydney on 40m) but I'm sure others will pop up in due course.

One spec. I have not seen is that for the minimum Windows machine on which to run VARA. As electrical power is often an issue for me I like to use my XP Intel Atom netbook when possible but I suspect it may struggle with VARA?

Graham
 

James /  Tony ,

There  are two , well  actually  three versions  of  VARA 

HF  designed  for  2.4 Khz  ssb  channel 
FM designed  for  standard  voice  FM  VHF replacing  1200 baud packet **
FM  high speed ,  designed to  replace the  9600 baud packet **
** VARA modem is  point to  point only,  not  support  AX25L packet , but  will  provide  forwarding 
at much enhanced speeds 

Not advisable to deploy  FM versions  over HF as the  coding  is not as tolerant to frequency 
drift , as HF , ie,  FM modulation  retains the  spectrum  integrity ..modem remains ARQ and  fully adaptive 

Vara was actually  designed  and coded  on a   windows  XP  computer,  as  far as can be  ascertained ,  VARA has no minimum  PC requirements 

Guide lines  would be ,  

As long as the  PC reaches the  minimum requirements  for  windows XP  ,
the  modem  will run .

73 - 
Graham 

James_Clark@...
 

Thanks Graham.

It'll be a couple of months before I can do much field testing but I'll endeavour to do some semi-formal P3 - Vara comparisons.

I suspect it will be like comparing HF antennas; very difficult (next to impossible!) to establish a consistent and repetitive test environment but rather one gets a "feel" for the better antenna over time - hardly good engineering but I can't think of a better way.

Graham
 
Edited

James . 

One thing that's  missed ,is,  vara maintains  on air presence as  an ofdm  modem,  and remains so, for all  transfer speeds , that is actually  a  design milestone , whereas   other  systems , revert to mmfsk  in an attempt to  maintain throughput in adverse conditions, [ofdm uses  multiple  psk  carriers , mmfsk  is  single  tone] But sacrifice the robustness of the  ofdm  modem,  a real test  is to have  in band  jamming  at the  same  time , 

The low  on air  rate  of  30 baud  per carrier of  vara and  being adaptive to 'channel  dely' , actually is able to facilitate  reliable  multi-hop long range communications at  maximum speed , only  thing that missing are  the  odd  sun-spot's ..Im not sure , what the  distance  record  stands at now, the  system  is  widely deployed , EU<>USA on 7 MHz has been achieved at good  rates  from a  maritime  mobile  station ..point to point  I don't know ?

I think you  find it  works quite  well ,  you can  deploy the  HF version  over  VHF using fm or ssb , may well  give extended range  and  speed , the  fm  version provides  higher maximum  speed , but not the  very low s/n  of the  HF version ..may be worth looking  at  

A more challenging  test  would be  Vara and  P4  

73-Graham
g0nbd


James_Clark@...
 

Hi Graham

Noted and understood and an interesting strategy to maintain the multi carrier protocol at all times. HF conditions seem to have been pretty bad lately so I'll be interested to see how Vara works under them.

VHF? Not a chance I'm afraid, I am always well beyond VHF range - Oz is a *big* country with lots of space :)

>A more challenging  test  would be  Vara and  P4

AT A$2100 for a P4 I'll leave that test to someone else - my P3 bought secondhand for $500 is quite enough for me :)

Graham
 

Ok James 

That'a a lot  of 'Buck's for a  Bit'  hihi

At the  lower ranges , the  modem has better  s/n and  throughput ,  than  conventional  2 tone rtty 
the concept is quite novel. The lower the  single carrier bit rate , the  lower the  bit error rate , so by 
running at 37.5 baud  , the  errors  per carrier , are minimised , 

The 'out the box'  thinking , is in the  data  processing , the better the  s/n , the  lower the 
error rate ,  so the  higher can be the  ''compression'' , the higher the  ''compression'',  the 
higher the  volume of 'plain text' that  can be  transmitted , 

As the  s/n falls ,  the  more  robust the  data  processing  becomes, ie less ''compressed'' 'so over the  same  link
the  'plain text' transfer slows .. but as the  on-air  rate remains  'low' at  all  transfer rates
issue's round  path  quality/bit errors are minimised 

73-Graham