We divided the blog into two parts so that you’d have an easier time of it. If you haven't already read Part 1, take a look at your convenience. For those of you who prefer one long read, we also stitched it together that way in our white paper <link>.
|Features||Your technical requirements|
|Budgets and Pricing||Customization|
|Your Testing Requirements||Consultation|
List your top 3 to 5 must-have test generator features.
If you’re just getting started and aren’t quite sure what to ask for, we’ve included a short Features list below. If you need more assistance, we’d be happy to help, contact us.
In one column, jot down your “must have” requirements” and in the other column jot down a list of any “nice-to-have-but-not-absolutely-necessary” items that you’d like to have in your perfect world of testing.
Stay focused on the program features and test behaviors that you need now, but make sure the software is robust enough to accommodate future growth. You don’t want to start looking for (yet) another program because you’ve outgrown the current one.
If you already have an Excel file that includes a prioritized list of features and behaviors—great! Email the list to the vendor in advance of a call or demo. This should save both of you time and allow the vendor to determine whether their product matches your testing requirements.
- Question Banking
- Automatic Scoring
- Question types – list what you require
- Security – list what you require
- Media support
- Email notification
- Multiple language test content delivery
- …Add your own requirements to the list
If you are doing the initial research and reporting back to a committee or decision maker, it’s best to find out if there’s a budget or price range that you can work with.
There are lots of choices ranging from free to expensive. I’m assuming you’re looking for a test authoring application that is feature-rich, includes support, training
Most testing solutions worthy of your attention will have various pricing schedules depending on your requirements.
We don’t recommend starting a conversation with a vendor without having some idea about what you want and some idea as to how much you are willing to spend. Giving the vendor details is always helpful—saves time and signals that you are a serious buyer.
If your budgets are a bit tight, don’t be shy about asking the vendor if they offer monthly, quarterly or bi-annual payment options.
If you are
If you are new to online testing and have never gotten a testing system up and running, we recommend working with prospective vendors who are responsive and eager to help. For your part, explicitly defining your requirements and expectations will make it easier for these vendors to determine whether their products match what you’re looking for.
The better you prepare, the more efficient the implementation. If you are rushed you are likely to forget something and make mistakes—like choosing the wrong test maker software. It’s never a good idea to rush through the selection and deployment of a new testing system, especially if you are a mid- to
Right now, we are...
- Not testing, but we need to get something going very soon
- Not currently testing but plan on testing, beginning <date, time-frame>
- Thinking about it, doing research, no specific start date
- Doing paper-and-pencil testing and don’t like it.
- Doing paper-and-pencil testing, like it and want to continue doing it
- Looking for another testing app—our current one is no longer available
- Using a "home-grown", testing application that is outdated
- Wanting an application that we can own and host on our servers
Test Authoring: We want to…
- Create and print tests
- Create question banks and then...
- Print and distribute paper tests or…
- Publish and deliver tests over our local network or…
- Publish and deliver tests over our company’s intranet or…
- Publish and deliver tests over the internet or…
- Publish and deliver tests over the intra-internet and print tests
The following applies to those of you who are giving tests on a computer or tablet.
When we say “create” we mean that you are either typing your questions
This information is stored in a database on your personal computer, on a server, or the vendor’s server “somewhere” on the internet.
When we say “publish” we mean that you make a test available to members of your test-taking audience. The details regarding how and when that is done will vary based on the features and behaviors that the exam software supports.
In most cases, your test-taking audience’s eligibility to take a test depends on whether they have a user profile in the testing software’s database. How this is done and how this is managed will vary from test maker to test generator to exam builder.
Hosting your test generator
By “hosting” we mean, “where do you want your exam creation software installed?”. Your test generator software can sit on your internal equipment or it can be deployed on the vendor’s servers.
Where you place your test making software may influence the cost and any technical resources you may need to get your system up and running.
Which of these options represents you?
- We don’t know where we want to deploy our test generator software
- We want to host the software in our operations center
- We want the vendor to host the software on their servers
Our testing audience consists of…
- A group of <X> students, employees, trainees, pre-hires, clients
- I don’t know how many we’ll have, we’re just getting this program off the ground.
- Daily, monthly, quarterly, annually
- Don’t know yet
Number of individuals to be tested
- Less than 5 individuals at the same time
- More than [X] at the same time
- Don’t know yet
We are interested in purchasing
- A yearly subscription
- A “pay-for-it-once” (perpetual) license
Does the vendor charge for…
- Tests Created
- Tests Taken
- Number of Users
- Test Administrators
- Test Data:
If we need to modify the product …
- Does the vendor offer custom programming services?
- Can they migrate our test data to other 3rd party applications?
- Does the vendor offer a non-compete source code license option?
If you are the test administrator, author or trainer-person who is “not real technical”, we recommend involving IT. It's never a good idea to buy something and then "surprise" your IT department with your purchase.
If you need assistance discussing technical matters, the vendor should be more than willing to provide whatever technical assistance you need, by phone, email and/or online documentation.
- What versions of SQL Server are supported?
- Is AD/LDAP supported?
web servers aresupported?
- Windows 10 support?... and so on.
If you need assistance, contact us we’d be happy to help,
If the vendor’s technical support team is slow to
As mentioned earlier, but bears repeating, before launching into an online demo—assuming the vendor offers this service—I suggest that you first find out if the software meets your testing requirements and budget. This can all be figured out through an email exchange or phone calls. If you don’t like to chat on the phone, email the vendor your requirements list and asked for some preliminary pricing. Keep in mind that the price may change based on taking a deeper dive into your requirements.
If the vendor tells you they can fulfill your requirements and work within your budget, ask to schedule an online demo. Live, online demos are a great way of making sure that there is no disconnect between what the vendor said, the product performance and the price.
If there are other members of your team that you would like to involve in the online session, see if you can coordinate schedules and give the vendor some dates/times that work for you. You may also
To avoid disappointments later, we recommend asking the vendor if they offer customization services before committing to a purchase. If you have a specific customization request in mind, it never hurts to ask for some pricing guidelines. If it’s a “must-have” feature requiring customization, ask the vendor if they can provide you with an estimate. If it’s a “must-have” feature can you afford the customization, if not, you may need to keep
Here are some possible options for your consideration.
If your customization is deemed minor by the vendor, they may be willing to include it in the package if you commit to a purchase.
If your customization request is more complex, we recommend providing the vendor with a written specification. It doesn’t have to be fancy or technical, it just needs to be concise with enough detail—pictures are also acceptable—to enable the vendor to provide a quote that they can honor.
It’s always a good idea to live with your purchase for a while, get comfortable with it. You may find that you don’t really need the customization you thought you needed, and discover that you need some other
When choosing a vendor, look for someone who is going to be responsive and genuinely interested in helping you succeed. You want someone who will help guide you through the implementation process.
If you’re the “evangelist” who’s herding cats toward a more efficient method of testing, make sure the vendor has your back.