Each year 44CON attracts between 100-200 submissions. Some of these are excellent talks, some are average and some are, well, let’s just say that some are below average. In this blog post I’ll try to go through what happens when the CFP closes and to help you answer the immortal question, “Has my talk been accepted/rejected?”
Along the way I’ll announce our first accepts, and most importantly explain the why of our CFP process.
I have learned a lot about how to track malware and attackers in network traffic while developing and improving the network forensics tool NetworkMiner throughout the past 10 years. The primary purpose of NetworkMiner has always been to help incident responders and forensic investigators to do their job more efficiently. Even though NetworkMiner is my favourite tool for analysing PCAP files I’m still a regular user of other tools such as Wireshark, tshark, tcpdump, Argus, ngrep, tcpflow and of course CapLoader. However, incident response and forensic work is much more than just knowing what tools to use. It is more about knowing what data to analyze and why.
I will teach several of my favourite techniques for analysing intrusions, tracking criminals and doing threat hunting at the Network Forensics Training at 44CON. The participants will learn how to investigate intrusions and find forensic artefacts in a dataset of several gigabytes of captured network traffic. The training primarily focuses on practical analysis techniques for finding and tracing malicious actors, which involves a great deal of hands-on practice with finding evil in PCAP data.
The first day of training focuses on analysis using only open source tools. The second day primarily covers training on the commercial software from Netresec, i.e. NetworkMiner Professional and CapLoader. All students enrolling in the class will get a full 6 month license for both these commercial tools. This training is not only a unique opportunity to learn how to use NetworkMiner and CapLoader directly from the guy who develops them, it is also a great excuse to spend two full days playing around with PCAP files.
You can find more details about the training here.
We opened our Early Bird Tickets last week and kept it quiet to give people a chance to get them, then when we announced them on our mailing list they went within an hour!
If you missed your Early Bird Tickets, fear not. Our CFP is still open. If you have a great workshop or talk idea, don’t forget that accepted talks and workshops include free attendance (and in most cases cover travel and accommodation from anywhere in the world). We wrote a blog post on how to boost your chances here.
If you’re itching to buy a ticket, regular tickets open at 8am GMT on the 10th of March, and will be available in our online shop.
Every year 44CON has a Call For Papers (CFP). The CFP is run by a panel of about 10 people from various parts of the industry, predominantly based in the UK. The process and technologies used have changed over the years, most notably last year when we replaced our existing bespoke CFP system with HotCRP and implemented a weighted average scoring mechanism based on HotCRP voting results.
TL;DR – I want to speak at 44CON
Ok, then do these things to boost your chances:
Submit a workshop with your talk
Make it clear where else you’ve submitted and/or might/will submit
Include links to other talks you’ve done, video if you have it
Get your talk in early for a better chance of scoring higher
Be enthusiastic, tell us of any boundaries, problems or needs, and work with us, not against us
Understanding how the CFP works
The CFP is opened on a particular date for submissions. Everyone speaking in Track 1 or 2 must go through the CFP process. Track 3 (which is sometimes used for workshops) is a little more fluid, for reasons I’ll discuss later.
Scoring and voting
Scoring is as much an art as science, but you can improve your chances of speaking at 44CON.
Our panel votes on and scores talks out of 5, normally in several tranches. It varies by individual and not everyone votes on every talk. People provide comments and feedback, which we pass onto those submitting on request. On average we get between 200-400 submissions a year.
When the CFP is due to close, we push panel members along to review and score submissions, particularly if they haven’t yet been voted on.
Once voting is complete, we divide the sum of scores from each voter by the number of voters to get an average, with an option for discussion under certain circumstances that can weight a score by up to + or – 0.5.
UK submissions normally get up-voted (with some exceptions, see below), and in the selection rounds there’s a strong bias towards UK-based speakers over non-UK-based speakers with identical scores, unless the non-UK-based speaker’s talk is exceptional in other ways. This doesn’t mean that your talk will be rejected if you’re not based in or from the UK. Non-UK based speakers make up the majority of our speakers, but there is definitely a small “home bias” amongst the panel members based in the UK.
Why does it take so long to find out if I’m accepted?
If you’re not sure what’s happening, contact us and we’ll give you an update.
Once we have the results we look to fill a specific number of slots, which varies each year. Acceptance messages are sent out in tranches, and when people return the speaker agreement, they’re confirmed. We normally send rejections for very low scoring talks, but there’s a glut of talks usually falling between 3.5-4.0, where they might be accepted if others scoring higher can’t make it.
If you’ve scored an average of 5.0, you’re pretty much guaranteed a slot and we’ll get in touch straight away. The bulk of submissions tend to hover around a score of 3-4, and 4.5 is normally the cut-off point for the first tranche of accepts. We then wait till the first tranche come back, or if we get no response, chase them up twice before moving on.
For the slots that free up, we move down the list, ensuring those who scored highest get picked first. Once we’ve filled up tracks 1 and 2, we move on to track 3.
After the first round of talk triage, cut-off tends to happen around average scores of 4.25 – 4.0/5.What this means is that there are a lot of good talks that just don’t get accepted at 44CON because we don’t have the space to support them, even with a third track. More often than not a talk or workshop rejection from 44CON does not mean it sucks. Ask for feedback and we’ll share what we can.
Wait, isn’t 44CON a two-track conference?
All speakers dress like this when preparing submissions.
Yes and no. For several years we’ve run a hidden track under various names. This is because we’ve wanted to give our backup speakers a chance to speak if someone drops out, but we don’t want to risk slots emptying on the main tracks. Inevitably people drop out along the way, people who are allocated to track 3 move onto the main tracks and this leaves gaps that we have the option to fill.
Sometimes we’ll look back at the talks list and look to offer a spot to someone on the list, however sometimes it’s easier to go to people we know are definitely coming and see if they have something. This is a completely arbitrary decision affecting two slots a year at most, and more often than not, 10-20 people want the slots. We generally operate on a first come, first serve basis.
Hacking the process
Now you know how the process works, let’s look at how you can subvert it to ensure your talk has the best chance of scoring high. Each voter on the panel is different, but there are certain things that, on average, will result in you being more favourably considered.
Submit both Talks and Workshops
We have 2-3 tracks to fill with talks, and get on average 200-400 submissions a year. We get less than 20 workshop submissions a year. Workshops are 2 hours long and come with an extra night’s accommodation when talks are also submitted.
If you want to maximise your chances of speaking at 44CON, submit a workshop.
Workshops are typically more intimate affairs with room for about 30-50 people sitting down, although we have had workshops with 100 people. If you’re not sure what to do in a workshop, imagine that if your main talk is about the theory, try a play-along walkthrough on how to do this in practice.
Every year we’ve run a formal CFP process, we’ve treated people who submit workshops far more favourably than people who submit talks alone. Even if your workshop is unrelated to your talk, both are likely to be up-voted considerably.
I cannot stress this enough. If you want to maximise your chances of speaking at 44CON, submit a workshop.
This only works if you submit your workshop separately to your talk. People submitting a talk and workshop in one don’t get the voting benefit separate talk and workshop submissions do. Finally, if you’re only prepared to come if your talk is accepted, please say so on both submissions.
Tell us where else your talk has been submitted
44CON is usually among the first events in the calendar after BlackHat and DefCon. Everyone wants to speak in Vegas, we understand that. Some people score BlackHat and DefCon talks slightly lower in order to give preference to newer talks, some don’t. It’s down to the panel. If you don’t tell us you’re talking at BlackHat or Defcon, and we find out by checking the site, panel members will remember next year and it may affect future submissions.
If you’re doing your reveal in Vegas, focus on your process at 44CON.
Not everyone in the UK can go to BlackHat or Defcon, so there’s not a massive deal in your talk being done in the UK afterwards. We do need to know what will be different. It takes a lot of effort to deliver a big Vegas talk, and making something different may seem like an awful lot of effort, but there’s an easy workaround that normally gets big bounces.
If you’re doing your reveal in Vegas, focus on your process at 44CON. If you spent 6 months trying to reverse engineer and compile code for an arcane architecture, we want to know how you went about it. We also appreciate failures as much as successes. Some of our better talks have been talks about how people have failed and what they learned.
If your talk is 70% different to your Vegas talk, say so. If it’s 50%, say so. If it’s 30%, say so. If you say so, and it’s not, then reviewers will know next year.
Show us your other talks
A picture speaks a thousand words, but a video of your talk lets the panel look at the type of speaker you are, how you approach your talks, and gives us an idea of where we think you might fit in best.
Showing us your other talks helps us fit you in.
This is an especially powerful tool for speakers coming from countries where English is a second language. All of our talks are delivered in English. We have some great speakers from across Europe, India and even China, and we want to keep the focus on the content, not on the way it’s conveyed.
It can be pretty scary delivering a talk in a second or third language, and it’s useful to see you speak, both to reassure voters when you’re delivering a talk, and to determine what help we might be able to offer if your research is brilliant, but you struggle with the language.
Even if you’re a native English speaker, throwing us a link to earlier talks lets us work out where and when we can put you. We often put more energetic speakers on in the afternoon for example.
Submit your talks early in the process
Most of the panel vote in several stages. Almost everyone votes for the first submissions coming in, and slowly dribble off after a while. At several points while the CFP is open, more people will vote, but because there are fewer talks to vote on, we’ve noticed that early talks score higher on average than those submitted later.
The more votes you get, the better the chance of bringing your voting average up and the better the chance of your talk being accepted. Submitting early gets you more (and often higher) voting scores.
Remember It’s A Two-Way Street
We completely understand how much of an effort you put in to come to speak at 44CON. Many of the crew talk at conferences themselves, and understand that you’re giving your time for free to go and speak at an event. That’s why we try to make the talk as cost neutral as possible for you to come and present. When people interact more with the event, and try to get involved, they’re generally more likely to have more positive responses.
There are certain speakers who come back to 44CON regularly such as Jerry Gamblin, Saumil Shah and Joe Fitzpatrick amongst others, all of whom make really strong efforts to interact with the crew and those attending. If, in your submission, you come across like you’re treating 44CON as just another con to shop the same talk around and disappear, you’re probably going to score lower than someone who comes across as though they really want to be there.
Coping with rejection
Our scoring method is not without its faults. No scoring system is perfect, and we’ve had to break bad news to big names as well as people with talks some of us thought were brilliant fits for the event.
If your talk was rejected, it’s not an indictment of you or your talk.
To help you deal with the sting of rejection, remember this:
Your talk not being accepted at 44CON does not mean we thought it was bad.
You absolutely have the right to ask for feedback. It might take a while depending on when you ask, but Steve will personally write back to you with as much detail as he can provide.
We’re all here to learn. If you think that we’ve made a mistake, or have ideas on how we can improve (beyond “accepting my awesome talk next time, dumbasses”), then we want to know.
Most importantly, your talk not being accepted does not mean we don’t want you to come and enjoy 44CON. We absolutely do want you to come, and will happily offer you a discount on a ticket as a thank you for submitting.
We want everyone to have a good time at 44CON. If you have any special needs or requests, from assistance with disabilities to being able to bring your kid(s) along just let us know. Unless it’s something we absolutely cannot accommodate, it will have no bearing on your submission’s consideration.
