I hope the people who attended the webinar enjoyed it. I certainly did, specially after we started the presentation and my level of anxiety went down a little

I won’t go into the details of the presentation since you can see all the slides bellow or you can see a recording of the presentation, co-sponsored and hosted by Sogeti.

I did wanted to go over some of the questions I was asked at the end of the presentation, since I found them interesting and worth reviewing a little more in detail.


Question: Do you test during the iteration on the cloud or do you test on in-house servers?  When do you test on the cloud, throughout the iteration or only during regression testing?
Answer: We believe in testing as soon as possible on servers that closely reflect our production environment, that is why all our tests (sanity, functional, regression, etc) are performed on servers and environments hosted on the cloud and using infrastructure similar (or identical) to the one we use to run our products live.
This is always a good practice, but it is vital for SaaS and cloud-based applications, where even a small issue in the configuration of the servers can make very big differences in the behavior of the system and the user-experience of our customers worldwide.


Question: SaaS applications are typically highly configurable i.e. different clients have different options and pages etc.  How would you approach this in terms of regression testing? Do you pick a sub-set of the most widely used pages, and create a dummy client configuration or what would you suggest as a best approach?
Answer: This is truly one of the challenges of testing a configurable and modular QA Management Solution such as PractiTest where our users not only can choose to use different modules of the system but can also configure it on different ways to match their needs.
I find that the best way to handle this challenge is by working with User Profiles, and having different environments set up ahead of time that reflect real-life configurations and customizations of our users.  This approach allows us to work on testing environments that reflect the whole spectrum of configurations and customizations we may have.
We also use a learning approach where on the occasions a new issue is detected on our Production environments we learn from it and find ways to incorporate it into our testing process, many times this involves adding a customization pattern to one of our existing profiles.


Question: Are there any specific challenges associated with being on the cloud (Amazon in your case) in regards to functional or load & performance testing?
Answer: There were some challenges involved with our initial testing steps in the cloud.  We’ve learned a lot and constantly improve the ways we work as we run into new difficulties and learn from them.  For example, I already said that our testing environments sit on the cloud, at the beginning it was very difficult to update these environments with new builds or even specific patches we wanted to test.  With time we realized we needed “something” that would allow the QA to deploy the product on the testing environment without the constant help of Development or IT; so we worked with them to create a set of scripts that would allows us to deploy to our testing environments automatically.
And this is obviously only an example of the many things we now do differently due to our testing environment constraints.


There were some additional questions that were asked on the webinar, but I will leave them for future posts since I believe they require more detailed attention.

In the meantime I wanted to thanks SoftTest & Anne-Marrie for inviting me, Sogeti for providing the logistical help and the hosting, and obviously all the people who attended for their time and their interesting questions and feedback.

If someone has additional questions from the presentation or the slides you can post them as comments and I will answer them gladly!

Related posts:

1. Leveraging Customers Rejections into Your Testing Environment
2. There really is a difference to CAST
3. How we test our SaaS QA Platform

What we like most about the magazine is that it’s fresh, cool and different.  Starting from the large number of new names who published their articles and all the way to the fun stuff such as the tester cartoons by Andy Glover, the blogs and even the conversations that are included inside inside.

A lot of people worked hard to make it happen, from the authors who submitted the articles and the rest of the pieces, and all the way to Rosie and Rob who literally spent days and nights to get it published.

THANKS, YOU DID A GREAT JOB!

Now I can’t wait to see what will come out in the next edition…

Related posts:

1. Testing Experience Magazine – Second Edition

It’s not uncommon to enjoy a good presentation from the audience perspective.
I mean, you can find the subject of the presentation interesting and new, you can enjoy the way the presenter conveys the message, you can even relate to the person standing in front of the room and feel good only because it could be you instead of him conveying the same “intelligent message” to the audience.

But yesterday I had one of those special occasions when you feel good with the presentation and its dynamic from the Presenter perspective.  And don’t get me wrong, it is not that I have not had good presentations before, but yesterday it felt something slightly better; it was one of those sessions when you feel that you didn’t provide a presentation so much as you were able to facilitate a group discussion around a specific topic.

The subject was not new or particularly exhilarating, what more can you say or discuss about Bugs in Software Testing?  But mainly because of this reason and due to the dynamic I tried to carry throughout the session I think we managed to take this subject and instead of reviewing it from 20,000 feet in 45 minutes, we examined specific points of it with microscopic precision and based on the points of view and experience from some pretty sharp people (some of whom I knew previously, but other who I didn’t and was glad to meet for the first time).

So I wanted to thank Alon and guys from SIGiST and Sela for organizing the event; but I also wanted to thank the people who took part of it and made it a fun and learning experience for all of us.  Gil, Gaby, Tal, Yoav, Yaron, Avi, and the rest of the participants who I am obviously forgetting… thanks for the time and the experience.

Also, as promised, I am posting the presentation in the QA Resources page of our site in case you want to take a look at it.

Related posts:

1. There really is a difference to CAST
2. Improving the efficiency by keeping track of your waste

Let’s start with the QA&Test conference in Bilbao, Spain.
One of the warmest conferences I’ve been as a presenter; in-spite of the rain and cold weather outside in the city, the sessions and the hallways were a great and warm place to meet fellow testers and exchange information and knowledge.

Guggenheim Museum in Bilbao

If you are up to the challenge and understand Spanish you are welcome to check this short interview they did of me (Mae, I was expecting you to cut the bad parts…)
In short, QA&Test is a truly recommended conference.

Then I got back to the office where we really turned up the heat on our development (& testing!) process in order to release PractiTest’s latest update that included the first version of our Jira integration.  By popular demand (of a large number of our users!) this was the first integration we created.
So for all you Jira fans out there, come and get it!!!

The third update is that I am starting to get involved in a very interesting test automation project taking place on a company where Agile is not only their newest fad, but where they are really leveraging this philosophy in order to change the way they develop (and test!) software.
So, expect to read some stuff coming from that arena soon.

And finally the invitation.

I am honored to have being invited to provide a Mini-Track session on the upcoming Israel-SIGiST meeting taking place on November 24th.

The name of my session is “A bug is only a bug, right?  Wrong!” and I promise to provide some food for thought around the things we do and some we don’t with one of the main outputs of our work as testers, our bugs.

Related posts:

1. Testing Intelligence in QA&Test ’09 – Bilbao

The conference started yesterday with some tutorials in the morning and presentations in the afternoon, where among other things I presented the concept of Testing Intelligence, with good comments and approvals from the session participants.

I was happy to see that concepts and approaches similar to Testing Intelligence are been developed in parallel by multiple QA Specialist who, same as myself, are realizing that Testing is not only about coverage, bugs, and AUT’s; but about information, stakeholders, and working within a team focused on developing a product.
One of these people is Derk-Jan de Grood from the The Netherlands who gave a great tutorial about Results Driven Testing, with some parallel lines of thought to the ones around Testing Intelligence.

I’m looking forward to a couple of more days of great sessions, starting today with a keynote from Mary Poppendieck…  Will post more in the following days, so stay tuned

Related posts:

1. Coffee, Nature & Software Testing…
2. Testing Intelligence™ – your deliverable and product as a QA Profesional
3. Testing Intelligence, a case for Intelligent Testing

I’m actually in Costa Rica, as some of you might know I was born & raised here so this is kind of a friends & family trip but not only that.
The last time I was here a friend commented about the growing local software development industry and this got me thinking, so a couple of months ago I started doing a little research and decided to put together a half-day seminar around Testing Intelligence and some additional topics making waves in the world of testing today.

The “event” took place this last Friday and I was really surprised to see the amount of people who showed up.  We got to fill the conference room with close to 50 people (Testers, Managers, Developers, Analysts, etc) from about 20 different companies (Banks, Development Firms, Outsourcing Companies, Consultants, etc) who were all interested in learning more about how to develop their products more effectively by applying their Tests in a more focused and coordinated way.

For me it was a blast!  To be able to talk about Testing Intelligence and other testing topics with people from Costa Rica was really great; specially to understand that here, in a country that up to now I had linked only to Great Coffee and Breath-Taking Nature there is also a growing community of testers who share the same dilemmas, challenges and issues that I get to see all over the world.

I am really hopping to continue contributing and working with this testing community to help it grow and expand.  As I told them during the session, they are keeping awfully quite and they need open-up to online communities and all the additional sources of information around the Internet…

So here goes for Costa Rica!  Here goes for excellent coffee, for green tracks along the volcanoes & beaches, and for a growing community of IT professionals (and specially their testers!).

Lastly I wanted to thank all the people who helped out on the organization of the seminar, and specially to Melissa Castillo who quietly and firmly made everything happen smoothly.  THANKS!!!

Related posts:

1. Do you think software testing is still in the Dark Ages? Turn on the lights!!!
2. Testing Intelligence in QA&Test ’09 – Bilbao
3. What Software Testing can learn from Basketball, games are won with a Great Defense

My presentation itself went pretty well (I think) and many people agreed with the TI approach, no surprises there.  But the real value for me came from the people who initially thought the method was not doable (or even realistic!) for their workplaces since it involved to many changes and education of their management on this new approach.

Reviewing the subject together with them I got to understand that the best approach for deploying TI on large organizations is to do it gradually and from within, based on a slowly-progressing plan that would allow the organization to learn and adapt to it, enhancing the chances for success in these environments.

The Testing Intelligence paper and the presentation I gave are posted on the CAST2009 site.
If you get to read them and have any comments, please make sure to send them to me.

I also wanted to talk about my CAST experience.
I have been in many conferences in the past (both as a presenter and an organizer) and so I can point to some stuff that really made a difference on the impact and the value I got from CAST. I invite the people organizing these types of conferences in the future to contact the CAST Team and try to learn from them.

1. Achieving real constructive criticism via facilitated sessions.
The methodology at CAST is that each session (from the largest keynote to the shortest presentation) are divided into 2 parts:
The frontal presentation – where the presenter talks about his subject.
and
The open season – where the audience asks questions, bring forward comments, and even refute the whole argument based on their experience and points of view.

The open season is made possible due to the active moderation of a “trained facilitator” who is constantly making sure the session is kept alive and on a positive and constructive note.

As a presenter, it is a bit scary at first.
Only after you understand that the audience doesn’t want to take you down but wants to help you out, can you participate from the interaction and even enjoy it.

2. Real eye-level interactions between all the participants (and I mean all the participants!!)
It doesn’t really matter if you are a junior tester or a published expert, it always felt right to approach a person and talk to him or to take part in an open discussion (with cups of coffee or bottles of beer!).

Some of the biggest names in the world of testing took part in the conference (Caner, Bach, Weinberg, etc) and at no time did you feel that any of them showed nothing but the most sincere sympathy (or at least respect!) for everyone else.
I don’t know if it was the blend of informal activities or simply the attitude of the organizers, but it was amazing.

3. The feeling of a peer-organized conference.
Again, this was a “feeling” more than something concrete, but at no time did you sensed any hidden agendas or sponsor oriented activities that made you feel someone was trying to sell something.

I don’t think I saw a single person in the conference that was not related in one way or another with the art of testing or at least teaching testing, and this made a whole lot of difference.
A conference from testers, by testers and to testers… what a refreshing idea

There were also the regular things you get from any “large” conference: a constant brainstorm of amazing ideas, camaraderie and genuine will to share and learn, and the chance to expand your circle of friends and colleagues.

One last thing to mention about this CAST was the fact that it was a relatively small conference, apparently affected by the “current economic crisis” that made an important number of people not to come to the conference.
I don’t know if this affected the conference itself, or if it actually helped to create an environment of a close and connected family of testing professionals that would not have been achieved by a group twice as big.

In any case I wanted to give my sincere thanks to the CAST Organization Committee who invited me to participate and allowed me to take part in the event.  Having had the experience of organizing similar events I can appreciate their work and performance.

I would start thanking by specific names but this will only make me leave out some people out, so as I already thanked each of you on the last day of the conference I wanted to thank you again as a group, for the great work and for the successful conference you allowed me to participate in.

I hope to see you all again in CAST2010!

Related posts:

1. In the CAST conference this week
2. I had a really good time presenting at the SIGiST mini-track last night
3. The Value of Peer Reviews (& why we don’t use it more!)